Re: Printing dict value for possibly undefined key

[DL Neil via Python-list]

On 11/25/2023 3:31 AM, Loris Bennett via Python-list wrote:

Hi,

I want to print some records from a database table where one of the
fields contains a JSON string which is read into a dict.  I am doing
something like

   print(f"{id} {d['foo']} {d['bar']}")

However, the dict does not always have the same keys, so d['foo'] or
d['bar'] may be undefined.  I can obviously do something like

   if not 'foo' in d:
 d['foo']="NULL"
   if not 'bar' in d:
 d['bar']="NULL"
   print(f"{id} {d['foo']} {d['bar']}")

Is there any more compact way of achieving the same thing?



What does "the dict does not always have the same keys" mean?

a) there are two (or...) keys, but some records don't include both;

b) there may be keys other than 'foo' and 'bar' which not-known in-advance;

c) something else.


As mentioned, dict.get() solves one of these.

Otherwise, there are dict methods which collect/reveal all the keys, all 
the values, or both - dict.keys(), .values(), .items(), resp.


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Newline (NuBe Question)

[DL Neil via Python-list]

On 11/27/2023 12:48 AM, Chris Angelico via Python-list wrote:

On Sun, 26 Nov 2023 at 21:08, Michael F. Stemper via Python-list
 wrote:


On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote:

Grizz[l]y,

I think the point is not about a sorted list or sorting in general It is
about reasons why maintaining a data structure such as a list in a program
can be useful beyond printing things once. There are many possible examples
such as having a list of lists containing a record where the third item is a
GPA for the student and writing a little list comprehension that selects a
smaller list containing only students who are Magna Cum Laude or Summa Cum
Laude.

studs = [
["Peter", 82, 3.53],
["Paul", 77, 2.83],
["Mary", 103, 3.82]
]


I've seen Mary, and she didn't look like a "stud" to me.



That's what happens when you abbreviate "student" though :) Don't
worry, there's far FAR worse around the place, and juvenile brains
will always find things to snigger at, usually in mathematical
libraries with "cumulative" functions.


The OP used an abbreviation: "studs". Why? Too lazy to type the full 
word? Abbreviation has full-meaning in the (narrow) domain? Was wanting 
something funny, or to snigger over?


Was the respondent sniggering? Perhaps he, like the OP, was also saving 
typing-time by making a joke, hoping that the OP would see the 
implicit-error in expecting others to understand that "studs" meant 
"students"?


Actually, Peter, Paul, and Mary were a band 
(https://www.peterpaulandmary.com/), so "studs" is even less expressive 
when the data also tells a story...


Working with "trainees", I avoid the word "student" even though some 
might see them as synonyms. In my mind, the abbreviation did not readily 
expand to the full word (mea culpa).


Accordingly, would not pass Code Review!
For the want of a few characters...
(https://en.wikipedia.org/wiki/For_Want_of_a_Nail)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Newline (NuBe Question)

[DL Neil via Python-list]

On 11/27/2023 1:08 AM, Roel Schroeven via Python-list wrote:
I prefer namedtuples or dataclasses over tuples. They allow you to refer 
to their fields by name instead of index: student.gpa is much clearer 
than student[2], and makes it less likely to accidentally refer to the 
wrong field.


+1
readability/comprehension!

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Newline (NuBe Question)

[DL Neil via Python-list]

On 11/27/2023 10:04 AM, Peter J. Holzer via Python-list wrote:

On 2023-11-25 08:32:24 -0600, Michael F. Stemper via Python-list wrote:

On 24/11/2023 21.45, avi.e.gr...@gmail.com wrote:

Of course, for serious work, some might suggest avoiding constructs like a
list of lists and switch to using modules and data structures [...]


Those who would recommend that approach do not appear to include Mr.
Rossum, who said:
   Avoid overengineering data structures.

   ^^^

The key point here is *over*engineering. Don't make things more
complicated than they need to be. But also don't make them simpler than
necessary.


   Tuples are better than objects (try namedtuple too though).


If Guido thought that tuples would always be better than objects, then
Python wouldn't have objects. Why would he add such a complicated
feature to the language if he thought it was useless?

The (unspoken?) context here is "if tuples are sufficient, then ..."



At recent PUG-meetings I've listened to a colleague asking questions and 
conducting research on Python data-structures*, eg lists-of-lists cf 
lists-of-tuples, etc, etc. The "etc, etc" goes on for some time! 
Respecting the effort, even as it becomes boringly-detailed, am 
encouraging him to publish his findings.


* sadly, he is resistant to OOP and included only a cursory look at 
custom-objects, and early in the process. His 'new thinking' has been to 
look at in-core databases and the speed-ups SQL (or other) might offer...


However, his motivation came from a particular application, and to 
create a naming-system so that he could distinguish a list-of-lists 
structure from some other tabular abstraction. The latter enables the 
code to change data-format to speed the next process, without the coder 
losing-track of the data-type/format.


The trouble is, whereas the research reveals which is faster 
(in-isolation, and (only) on his 'platform'), my suspicion is that he 
loses all gains by reformatting the data between 'the most efficient' 
structure for each step. A problem of only looking at the 'micro', 
whilst ignoring wider/macro concerns.


Accordingly, as to the word "engineering" (above), a reminder that we 
work in two domains: code and data. The short 'toy examples' in training 
courses discourage us from a design-stage for the former - until we 
enter 'the real world' and meet a problem/solution too large to fit in a 
single human-brain. Sadly, too many of us are pre-disposed to be 
math/algorithmically-oriented, and thus data-design is rarely-considered 
(in the macro!). Yet, here we are...


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Newline (NuBe Question)

['DL Neil' via Python-list]

Avi,

On 11/27/2023 4:15 PM, avi.e.gr...@gmail.com wrote:

Dave,

Back on a hopefully more serious note, I want to make a bit of an analogy
with what happens when you save data in a format like a .CSV file.

Often you have a choice of including a header line giving names to the
resulting columns, or not.

If you read in the data to some structure, often to some variation I would
loosely call a data.frame or perhaps something like a matrix, then without
headers you have to specify what you want positionally or create your own
names for columns to use. If names are already there, your program can
manipulate things by using the names and if they are well chosen, with no
studs among them, the resulting code can be quite readable. More
importantly, if the data being read changes and includes additional columns
or in a different order, your original program may run fine as long as the
names of the columns you care about remain the same.

Positional programs can be positioned to fail in quite subtle ways if the
positions no longer apply.


Must admit to avoiding .csv files, if possible, and working directly 
with the .xls? original (cf expecting the user to export the .csv - and 
NOT change the worksheet thereafter).


However, have recently been using the .csv format (as described) as a 
placeholder or introduction to formatting data for an RDBMS.


In a tabular structure, the expectation is that every field (column/row 
intersection) will contain a value. In the RDBMS-world, if the value is 
not-known then it will be recorded as NULL (equivalent of Python's None).


Accordingly, two points:
1 the special case of missing/unavailable data can be handled with ease,
2 most 'connector' interfaces will give the choice of retrieving data 
into a tuple or a dictionary (where the keys are the column-names). The 
latter easing data-identification issues (as described) both in terms of 
improving over relational-positioning and name-continuity (or column 
changes/expansions).



The point about data 'appearing' without headings should be considered 
carefully. The phrase "create your own names for columns" only vaguely 
accesses the problem. If someone else has created/provided the data, 
then we need to know the exact design (schema = rules). What is the 
characteristic of each component? Not only column-names, but also what 
is the metric (eg the infamous confusion between feet and meters)...




As I see it, many situations where some aspects are variable are not ideal
for naming. A dictionary is an example that is useful when you have no idea
how many items with unknown keys may be present. You can iterate over the
names that are there, or use techniques that detect and deal with keys from
your list that are not present. Not using names/keys here might involve a
longer list with lots of empty slots to designate missing items, This
clearly is not great when the data present is sparse or when the number of
items is not known in advance or cannot be maintained in the right order.


Agreed, and this is the draw-back incurred by folk who wish to take 
advantage of the schema-less (possibility) NoSQL DBs. The DB enjoys 
flexibility, but the downstream-coder has to contort and flex to cope.


In this case, JSON files are an easy place-holder/intro for NoSQL DBs - 
in fact, Python dicts and MongoDB go hand-in-glove.



The next issue raised is sparseness. In a table, the assumption is that 
all fields, or at least most of them, will be filled with values. 
However, a sparse matrix would make such very 'expensive' in terms of 
storage-space (efficacy).


Accordingly, there are other ways of doing things. All of these involve 
labeling each data-item (thus, the data expressed as a table needs to be 
at least 50% empty to justify the structural change).


In this case, one might consider a tree-type of structure - and if we 
have to continue the pattern, we might look at a Network Database 
methodology (as distinct from a DB on a network!)




There are many other situations with assorted tradeoffs and to insist on
using lists/tuples exclusively would be silly but at the same time, if you
are using a list to hold the real and imaginary parts of a complex number,
or the X/Y[/Z] coordinates of a point where the order is almost universally
accepted, then maybe it is not worth using a data structure more complex or
derived as the use may be obvious.


No argument (in case anyone thought I might...)

See @Peter's earlier advice.

Much of the consideration (apart from mutable/immutable) is likely to be 
ease of coding. Getting down 'into the weeds' is probably pointless 
unless questions are being asked about (execution-time) performance...



Isn't the word "obvious" where this discussion started? Whereas "studs" 
might be an "obvious" abbreviation for "students" to some, it is not to 
others (quite aside from the abbreviation being unnecessary in this 
day-and-age).


Curiously, whereas I DO happen to think a point as ( x, y, ) or 

Re: "Edit With Python" option missing

[DL Neil via Python-list]

Hi Aakash,

On 31/08/19 8:54 PM, Akash verma wrote:

"Edit With Python" option missing from message context when right clicked
with mouse .


There are two stages to working with a Python program: editing the 
source-code, and executing the program.


For the first, most use a text editor/IDE, eg Sublime Text, PyCharm. 
Although Python comes with "Idle" as a built-in editor, plus the REPL. 
Accordingly, you will want to set the (right-click) context menu for the 
.py file-type/extension to open your chosen editor (if it's not already).


In the second case, it depends very much upon which operating 
system/editor combination you are using. Most of the better editors have 
a 'magic button' to do the job. In case you are using MS-Windows (I 
don't!) please review https://docs.python.org/3.3/using/windows.html


In the case of executing a program without editing the code, most of us 
utilise 'the command line', eg


python3 source.py


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Hi how do I import files inside a txt file?

[DL Neil via Python-list]

On 3/09/19 1:48 AM, Spencer Du wrote:

On Monday, 2 September 2019 15:29:07 UTC+2, Joel Goldstick  wrote:

On Mon, Sep 2, 2019 at 9:21 AM Spencer Du  wrote:


On Monday, 2 September 2019 15:03:52 UTC+2, Joel Goldstick  wrote:

On Mon, Sep 2, 2019 at 8:46 AM Spencer Du  wrote:


On Monday, 2 September 2019 13:36:06 UTC+2, Pankaj Jangid  wrote:

Spencer Du  writes:


How do i import files inside a txt file if they exist in the current directory?

Here is the current code but I dont know how to do it correctly.

import paho.mqtt.client as mqtt
from mqtt import *
import importlib
import os
import os.path
# from stateMachine import *

with open("list_of_devices.txt", "r") as reader:
 for item in reader:
 try:
 os.getcwd()
 print("hi")
 except:
 print("error")

This is "list_of_devices.txt":
test1,test2

Each name refers to a python file.


My interpretation is that you want to read a file (list_of_devices.txt)
and this file contains names of other files and you want to read those
files as well and do something with them (read or print or whatever).

You can approach it like this: write a function to read a file and work
on it. Like this,

def fn(fname):
 with open(fname, "r") as f:
  try:
 # work with f
  except:
 print("error")

Then use this function in your code that you have writen. Like this

with open("list_of_devices.txt", "r") as reader:
  for item in reader:
  try:
 fn(item)
  except:
 print("error")

In the example that you gave, you have written contents of
"list_of_devices.txt" as

test1,test2

Take care to read them as comma separated. Or if you have control then
write them on separate lines.

Regards.
--
Pankaj Jangid


Hi Pankaj

I dont understand so what is complete code then?

Thanks
Spencer
--
https://mail.python.org/mailman/listinfo/python-list


Pardon me for guessing, but your question seems to imply that you know
how you want to do something .. but I'm not sure you have tackled your
problem correctly.

My guess is:  Depending upon the names listed in a text file, you want
to do different imports into your program.   You don't yet know how to
read a file with python.

First, when you run your program, python compiles it in order.  Since
you don't know what you want to import until after you run your
program, you can't import those modules.  You may be able to run a
program to read the module list, then have it output to a new file the
code you eventually want to run based on the modules you discovered.
That sounds cute in a way, but probably not in a good way.  You could
also surround import statements with try/except code that will import
what it can, and alert you when it can't

Can you give us the bigger picture of what you want to accomplish?
This might lead to a better solution than the one you are thinking of
now

--
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays


Hi

I have a txt file which contains the names of files. They are .py files. I want 
to import them into a python file if they exists in current directory and if 
the name of file does not exist then print out error and not import. How do I 
do this?

Thanks
Spencer


Here is a discussion on Stack overflow that lays out how you can
dynamically import files.  This should get you started in the right
direction.  First, see if you can write code to read the file, and
retrieve the names of the modules you want to import.   Come back if
you stumble with your code for that

https://stackoverflow.com/questions/301134/how-to-import-a-module-given-its-name-as-string

--
https://mail.python.org/mailman/listinfo/python-list




--
Joel Goldstick
http://joelgoldstick.com/blog
http://cc-baseballstats.info/stats/birthdays


Ok I have this code to retrieve the names of modules I want to import. Now how 
do I check if they exist in current directory and if they exist import them 
into the python program. Thanks.

with open("list_of_devices.txt", "r") as f:
for item in f:
print(item)



Perhaps it is time to slow-down and take a breather?

The answer to this step has already been covered in this thread!


Many languages require one to *anticipate* every contingency - anything 
that could go 'wrong'. For example, that a file does not exist and 
cannot be imported.


However, Python follows a philosophy that it is "easier to ask 
forgiveness than it is to get permission"*. In other words, to use the 
try...except construct - in this case, to attempt an import (the "try") 
and if that fails (probably because the file does not exist) then to 
defend/react accordingly (the "except" - "except" = "exception").


In other words, only worry about 'the problem' (post fact), should it 
arise. This works quite neatly for your use-case.



Such is what is called "a Python idiom" - the way things ar

Re: How to remove a string from a txt file?

[DL Neil via Python-list]

On 5/09/19 3:08 AM, Spencer Du wrote:

Hi

I want to remove a string from a txt file and then print out what I have 
removed. How do I do this.

The txt file is in this format and should be kept in this format.

txt.txt:
laser,cameras,


Is this a homework assignment?

What code do you have so far?

How to identify the beginning of the sub-string to be removed?

How to identify the end of the sub-string?

How does one "remove a string" AND "kept in this format"?

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Formatting floating point

[DL Neil via Python-list]

On 5/09/19 5:12 AM, Dave via Python-list wrote:
...

My question is why, and where do I find a reliable source of information 
on formatting numbers?  Not interested in replacement values like '{} 
{}'.format(1, 2).



Agreed: there's ton(ne)s of information 'out there', much of it old, eg 
Python2, "formatter" (deprecated since v3.4), methods pre-dating 
f-strings, etc; and more of it rather casually thrown-out-there. 
Learning from StackOverflow (etc) has its limits/perils!


Authoritative Python docs: https://docs.python.org/3/
The "Mini-Language": 
https://docs.python.org/3/library/string.html#formatspec


Current advice (v3.6+) is to study the newer f-strings (formally 
"formatted string literals"): 
https://docs.python.org/3/tutorial/inputoutput.html which came from PEP 
498: https://www.python.org/dev/peps/pep-0498/


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Finding lines in .txt file that contain keywords from two different set()

[DL Neil via Python-list]

On 9/09/19 4:02 AM, A S wrote:

My problem is seemingly profound but I hope to make it sound as simplified as 
possible.Let me unpack the details..:


...


These are the folders used for a better reference ( 
https://drive.google.com/open?id=1_LcceqcDhHnWW3Nrnwf5RkXPcnDfesq ). The files 
are found in the folder.



The link resulted in a 404 page (for me - but then I don't use Google). 
So, without any sample data...


> 1. I have one folder of Excel (.xlsx) files that serve as a data 
dictionary.

>
> -In Cell A1, the data source name is written in between brackets
>
> -In Cols C:D, it contains the data field names (It could be in either 
col C or D in my actual Excel sheet. So I had to search both columns

>
> -*Important: I need to know which data source the field names come from
>
> 2. I have another folder of Text (.txt) files that I need to parse 
through to find these keywords.



Recommend you start with a set of test data/directories. For the first 
run, have one of each type of file, where the keywords correlate. Thus 
prove that the system works when you know it should.


Next, try the opposite, to ensure that it equally-happily ignores, when 
it should.


Then expand to having multiple records, so that you can see what happens 
when some files correlate, and some don't.


ie take a large problem and break it down into smaller units. This is a 
"top-down" method.



An alternate design approach (which works very well in Python - see also 
"PyTest") is to embrace the principles of TDD (Test-Driven Development). 
This is a process that builds 'from the ground, up'. In this, we design 
a small part of the process - let's call it a function/method: first we 
code some test data *and* the expected answer, eg if one input is 1 and 
another is 2 is their addition 3? (running such a test at this stage 
will fail - badly!); and then we write some code - and keep perfecting 
it until it passes the test.


Repeat, stage-by-stage, to build the complete program - meantime, every 
change you make to the code should be tested against not just 'its own' 
test, but all of the tests which originally related to some other 
smaller unit of the whole. In this way, 'new code' can be shown to break 
(or not - hopefully) previously implemented, tested, and 'proven' code!


Notice how you have broken-down the larger problem in the description 
(points 1 to 5, above)! Design the tests similarly, to *only* test one 
small piece of the puzzle (often you will have to 'fake' or "mock" 
data-inputs to the process, particularly if code to produce that unit's 
input has yet to be written, but regardless 'mock data' is thoroughly 
controlled and thus produces (more) predictable results) - plus, it's 
much easier to spot errors and omissions when you don't have to wade 
through a mass of print-outs that (attempt to) cover *everything*! (IMHO)


Plus, when a problem is well-confined, there's less example code and 
data to insert into list questions, and the responses will be 
equally-focussed!



Referring back to the question: it seems that the issue is either that 
the keywords are not being (correctly) picked-out of the sets of files 
(easy tests - for *only* those small section of the code!), or that the 
logic linking the key-words is faulty (another *small* test, easily 
coded - and at first fed with 'fake' key-words which prove the various 
test cases, and thus, when run, (attempt to) prove your logic and code!)



--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

WedWonder: Scripts and Modules

[DL Neil via Python-list]

In this day-and-age do you have a script in live/production-use, which 
is also a module? What is the justification/use case?


(discounting distutils and similar installation tools, or unit testing 
methodology)



There are over 500 questions on StackOverflow which refer to Python's

if __name__ == __main__:

construct. Even more if you include the idea of a main() multiple 
entry-point.


This construct enables code to distinguish between being "run" as a 
"script", and being imported as a "module". (which is not in question)


It intrigues me how often (per SO, quoted above) this causes anything 
from a pause to complete consternation amongst Python neophytes. In my 
own case, I accepted it as a "Pythonic" idiom, adopted its use in my 
scripts, and moved on.


Until I adopted unittest/pytest and took-on the closer definitions of 
TDD, I used to use the script/module switch as a means of testing 
modules (see also 'good, old days', below). These days, pytest requires 
a different approach and splits 'test code' from (production) "delivered 
code". Oh yeah!


However, I can't recall ever gaining similar benefit from using the 
'switch' within code designed to be a "script"!



Ages ago some beginner asked me the 'script/module switch' question, and 
I gave the standard/formulaic/Pythonic answer. However, reversing the 
question in my mind led me to ask (myself): how many scripts do I have 
(in "production use") which are ever used (also) as a module by some 
other script? I think the answer is/was: "none"! Accordingly, (spoiler 
alert: this statement may be heresy) I stopped using the "if".



Back in the ?good, old days of mainframes we wrote "monolithic" 
programs. Then we developed "modular programming" and started to split 
code into functional units, ie functions, subroutines, paragraphs, 
procedures. Thus "code re-use" was born. (see what a new idea it is! 
Subroutine libraries on mag-tape, anyone?) We distinguished a subroutine 
or "called code" (importable module in Python) from the calling-code by 
calling the latter the "main-line" (nothing to do with/say no to drugs). 
We even developed little "stub" programs; which would exercise or test 
specific subroutines to ensure that they worked (wow, how new is (much 
of) Test-Driven Development?)


So (putting nostalgia aside), these days my Python *scripts* are pure 
"main-line" code.



Faced with the perennial "main/name" question again yesterday, led me to 
review the above policy/laziness, and thus this "Wednesday Wondering":


- modules aside, how often do we write multiple-entry code these days, 
as opposed to having multiple scripts/main-lines which call re-usable 
functional modules, as-and-when?


I still don't have a single file containing a combination script/module 
amongst my applications. Do you? Why/who/how/when?


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: WedWonder: Scripts and Modules

[DL Neil via Python-list]

On 12/09/19 8:43 AM, Chris Angelico wrote:

On Thu, Sep 12, 2019 at 6:34 AM DL Neil via Python-list
 wrote:


In this day-and-age do you have a script in live/production-use, which
is also a module? What is the justification/use case?




Yes, absolutely. It's the easiest way to share code between two
scripts. Here's an example that I created recently:

https://github.com/Rosuav/shed/blob/master/BL1_find_items.py
https://github.com/Rosuav/shed/blob/master/BL2_find_items.py

These programs do similar jobs on very different formats of file, so
there's a small amount of common code and a large amount that isn't
common. One of the common sections is the FunctionArg class, which
ties in with argparse; it's implemented in BL1_find_items, and then
imported into BL2_find_items.

Of course I could break this out into its own dedicated file... but
why bother? It's not significant enough to warrant its own module, and
I don't see any value in an importable file of "all the stuff that I
might feel like importing"; it's just these two files that will need
this.

Basically, the script/module distinction is a convenient way to
simplify a common situation that doesn't need the overhead of anything
else. If the code starts getting used in lots more places, it'll
eventually get promoted to actual module.



Thanks for such a rapid response!

Interestingly, we appear to have opposite approaches to this situation - 
as soon as I think the word "common" it morphs immediately into "module".


I have also needed to scan a directory/tree recently, but took the 
course of refactoring a previous implementation's 'directory-walk' code 
into its own (generator) function, and thus the mainline of the newer 
application calls the utility function and chooses whether or not to 
deal with each 'found' file/dir in-turn. The previous script using the 
code was similarly refactored.
(and there are no doubt ?many more scripts 'lurking' in my code-base 
which could be similarly 'improved' - such frequent commonality leading 
to my preference).


My bias (I'm not criticising/complaining about/decrying the choices you 
have illustrated) probably comes out of "separation of concerns". An 
issue which has 'bitten' me, more than once...


For example both BL1 and BL2 feature:

def money(savefile): savefile.money[0] += 500 # Add more dollars

- a minor issue, agreed (and 'picking on it' purely to make a point).

The "s-o-c" is, that one day it will be possible to decide that the unit 
of addition should change, but only (remember to) amend the code in one 
of the two scripts!
(when it comes to trusting my memory, I'm an 'old git' with history, but 
not an old git repo with "history"!)


Which brings me back to the preference for 'encapsulation' (just to 
prove I can speak "OOP"), and why I would have written quite separate 
'main-lines' and extracted money() (and presumably a lot more) into a 
module which could then be called by both.



Aside1:
However, you've started me thinking about a related 
consideration/philosophy (Danger Will Robinson!) - but please let me 
cogitate on that for a day or so...


Aside2:
Meantime, thanks for the opportunity to review your code. I was thinking 
of considering "data classes" (new in v3.7, IIRC) in relation to an SQL 
interface on today's ToDo list - constructing the application's queries 
and the data transfers 'in' and 'out', without the 'weight' of 
SQLAlchemy. Perhaps?


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Email messages from grouped email using IMAPClient in Python.

[DL Neil via Python-list]

On 12/09/19 5:06 PM, Srinivas Pullabhotla wrote:

Hello,

I am trying to fetch email messages from a gmail inbox. So, there will be 1000s 
of messages sent to Inbox and since they are 1000s, the emails are grouped 100 
per each email item.

When I tried this method, the program only fetches some but not all and 
especially it happens with grouped email messages. How can I get all the 
messages from that grouped email.

import email, time, sys
 from imapclient import IMAPClient

 with IMAPClient(HOST) as server:
 server.login(USERNAME, PASSWORD)
 server.select_folder('INBOX', readonly=True)

 messages = server.search(['ALL', 'UNSEEN'])
 for uid, message_data in server.fetch(messages, 'RFC822').items():
 email_message = email.message_from_bytes(message_data[b'RFC822'])
 print(email_message.get_payload(None, True))


The program should fetch all the email messages from the grouped email and 
output to the file (in my case I am grabbing the href links). How best I can 
achieve this ? Appreciate thoughts and suggestions.



First debug is to assign the server.select_folder() return values, and 
subsequently print them for inspection. (know what you are dealing 
with/that you have been given something to work with)


The server.search() doesn't request ALL of the msgs. Are you sure those 
missing from the 'group' are not marked as 'read'? So, second debug 
would be to remove the 'UNSEEN' criteria and observe any difference.


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: WedWonder: Scripts and Modules

[DL Neil via Python-list]

On 12/09/19 10:37 AM, Alan Bawden wrote:

DL Neil  writes:


... However, reversing the question in my mind led me to ask (myself):
how many scripts do I have (in "production use") which are ever used
(also) as a module by some other script? I think the answer is/was:
"none"! Accordingly, (spoiler alert: this statement may be heresy) I
stopped using the "if".


I found that the problem with doing this is that `pydoc' now _runs_ my
scripts, when all I wanted was a quick look at the constants and procedures
defined in them.


I haven't experienced this. However, I'll make a note to test...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: WedWonder: Scripts and Modules

[DL Neil via Python-list]

On 12/09/19 10:59 AM, Cameron Simpson wrote:

On 12Sep2019 08:24, DL Neil  wrote:
In this day-and-age do you have a script in live/production-use, which 
is also a module? What is the justification/use case?


Many. Many many.

1: Many of my modules run their unit tests if invoked as the main 
programme.


I used to do this, but now prefer to keep tests in separate modules - 
and separate directories.



2: Several modules are their own utility programme. I've got a heap of 
these - anything that provides useful access to something can often be 
usefully used from the command line.


This is an very interesting idea - you're way ahead of me on this!

Would it be fair to describe these as more Python for systems 
programming/system administration, than an application for (simple) users?

(what other kind is there???)



Consider: if you write a package, would it have a __main__.py?

Well, if the answer is ever "yes" then the same applies to ordinary 
modules, simple enough to not be worth splitting onto a package.


May not properly appreciate this point...



So, yes, for me this is really really common.

Even for my current client project, which is largely a package, several 
of the individual modules within the package have their own main 
programmes for testing and for various utility tasks dealing solely with 
that particular subsystem. There's an overarching shell script to set up 
the environment and then do various things from the command line, and it 
directly invokes particular modules for some operations that act only on 
one subsystem.


Interesting!

Gives me pause for thought: perhaps I lean too heavily on putting 
'stuff' in the test routines (and view the application from that 
'direction' too often).


Multiple entry-point systems seem relatively unusual these days - 
perhaps a natural flow-on effect of the rise of gui- and menu-based systems?


With one client, over the years, we've developed a number of (basically) 
statistical analyses. Each analysis was born from a separate project. 
Each lives in its own directory (tree). There are some common modules 
held in a separate 'utility' library/package/directory. Whilst it has 
been suggested, the idea of an "overarching" menu-type top-level 
controller/distributor has never risen up the ranks of the "backlog" 
(sensible criticism: the money would be better spent on other 
priorities) - they (or at least the older ones) are quite happy to 
'drop' to the command line and type the required command+args. I think 
that's the closest I come to what you have described. (out of interest) 
Would you characterise it as a common project structure?


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: WedWonder: Scripts and Modules

[DL Neil via Python-list]

On 12/09/19 8:22 PM, Barry Scott wrote:




On 11 Sep 2019, at 21:24, DL Neil via Python-list  
wrote:

In this day-and-age do you have a script in live/production-use, which is also 
a module? What is the justification/use case?

(discounting distutils and similar installation tools, or unit testing 
methodology)


There are over 500 questions on StackOverflow which refer to Python's

if __name__ == __main__:

construct. Even more if you include the idea of a main() multiple entry-point.

This construct enables code to distinguish between being "run" as a "script", and being 
imported as a "module". (which is not in question)



In my mind this is a question about what side-effects of importing a module are
desireable and which are not.


Precise description!



A trivia script does not need the __name__ == '__main__' as its all about its 
side effects.

As scripts become more complex having it run on import might make debugging 
harder
and prevents reuse.


Is this an informal distinction: that modules are written for re-use but 
main-lines to be unique/single-purpose?




For example I will import a script at the REPL and examine it and call function 
in it to
help me understand and fix problems.  Having a __name__ == '__main__' is 
important
to allow this.


Why?

If we import sys (from the PSL, purely as an example), we don't 
expect/need any execution phase, can immediately follow (in the REPL) 
with help(sys) and similar, and can debug/explore from there:



import sys
help(sys)



sys.path
['', '/usr/lib64/python37.zip', '/usr/lib64/python3.7', 
'/usr/lib64/python3.7/lib-dynload', 
'/usr/lib64/python3.7/site-packages', '/usr/lib/python3.7/site-packages']




I often have modules that are part of a larger program that have their own 
main() functions
to unittest or give access to parsers etc.


and the main() is the sole content of the if __name__ etc structure?



In large projects with many modules import with side-effect can make for a 
maintenance
burden.



You seem to have unmasked an assumption under which I operate. (well 
done - you know what 'they' say about assumptions!)


As you say, (non-designed) side-effects are undesirable.

My 'rule' is that modules only contain definitions, eg classes and 
functions. Thus, *nothing* executes upon import.


During a code review, some eagle-eye, noticed (and questioned) I had 
re-factored some 'constants' which control a "switch" structure from out 
of the module-space, and into the class where they 'belonged', without 
having any other 'good reason'.



I have no recollection of the history or rationale for this 
policy/practice, nor can I remember whether it 'belongs to Python' or 
some other language from which I've imported it (hah, punny joke!) 
Perhaps it comes from my preference to from...import... rather than 
import...as... which largely eschews the module namespace?

(or perhaps I'm just a closet control-freak?)

That said, I'm sure there must be modules of my code which break this 
'rule', somewhere (certainly in the 'quick and dirty' department).


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Friday Finking: 'main-lines' are best kept short

[DL Neil via Python-list]

(this follows some feedback from the recent thread: "WedWonder: Scripts 
and Modules" and commences a somewhat-related topic/invitation to 
debate/correct/educate)



Is it a good idea to keep a system's main-line* code as short as 
possible, essentially consigning all of 'the action' to application and 
external packages and modules?


* my choice of term: "main-line", may be taken to mean:
- the contents of main(),
- the 'then clause' of an if __name__ == __main__: construct,
- a __main__.py script.


In a previous thread I related some ?good, old days stories. When we 
tried to break monolithic programs down into modular units, a 'rule of 
thumb' was "one page-length" per module (back in the mainframe days our 
code was 'displayed' on lineflo(w) (continuous stationery) which was 66 
- call it 60, lines per page - and back-then we could force a page-break 
where it suited us!). Then when we moved to time-share screens (80 
characters by 24 lines), we thought that a good module-length should 
conform to screen-size. These days I have a large screen mounted in 
'portrait mode', so on that basis I'm probably back to 50~60 lines (yes, 
these old eyes prefer a larger font - cue yet more cheeky, age-ist 
comments coming from my colleagues...)


Likely I have also picked-up and taken-to-heart the *nix mantra of code 
doing 'one job, and doing it well' (and hence the extensive powers of 
redirects, piping, etc - in Python we 'chain' code-units together with 
"import"). Accordingly, I tend to err on the side of short units of 
code, and thus more methods/functions than others might write.


In "Mastering Object-oriented Python" the author discusses "Designing a 
main script and the __main__ module" (Ch17):

<<<
A top-level main script will execute our application. In some cases, we 
may have multiple main scripts because our application does several 
things. We have three general approaches to writing the top-level main 
script:
• For very small applications, we can run the application with python3.3 
some_script.py . This is the style that we've shown you in most examples.
• For some larger applications, we'll have one or more files that we 
mark as executable with the OS chmod +x command. We can put these 
executable files into Python's scripts directory with our setup.py 
installation. We run these applications with some_script.py at the 
command line.

• For complex applications, we might add a __main__.py module in the
application's package. To provide a tidy interface, the standard library
offers the runpy module and the -m command-line option that will use 
this specially named module. We can run this with python3.3 -m some_app.

[explanation of "shebang" line - the second approach, above]

Creating a __main__ module
To work with the runpy interface, we have a simple implementation. We 
add a small __main__.py module to our application's top-level package. 
We have emphasized the design of this top-level executable script file.
We should always permit refactoring an application to build a larger, 
more sophisticated composite application. If there's functionality 
buried in __main__.py , we need to pull this into a module with a clear, 
importable name so that it can be used by other applications.

A __main__.py module should be something small like the following code:

import simulation
with simulation.Logging_Config():
with simulation.Application_Config() as config:
main= simulation.Simulate_Command()
main.config= config
main.run()

We've done the minimum to create the working contexts for our 
application. All of the real processing is imported from the package. 
Also, we've assumed that this __main__.py module will never be imported.
This is about all that should be in a __main__ module. Our goal is to 
maximize the reuse potential of our application.

[example]

We shouldn't need to create composite Python applications via the 
command-line API. In order to create a sensible composition of the 
existing applications, we might be forced to refactor stats/__main__.py 
to remove any definitions from this module and push them up into the 
package as a whole.

>>>

Doesn't the author thus suggest that the script (main-line of the 
program) should be seen as non-importable?


Doesn't he also suggest that the script not contain anything that might 
be re-usable?


Accordingly, the script calls packages/modules which are both importable 
and re-usable.


None of which discounts the possibility of having other 'main-lines' to 
execute sub-components of the (total) application, should that be 
appropriate.


An issue with 'main-line' scripts is that they can become difficult to 
test - or to build, using TDD and pytest (speaking personally). Pytest 
is great for unit tests, and can be used for integration testing, but 
the 'higher up' the testing pyramid we go, the less effectual it becomes 
(please don't sho

Re: python is bugging

[DL Neil via Python-list]

On 22/09/19 5:08 AM, Dave Martin wrote:

On Saturday, September 21, 2019 at 12:44:27 PM UTC-4, Brian Oney wrote:

On Sat, 2019-09-21 at 08:57 -0700, Dave Martin wrote:

On Saturday, September 21, 2019 at 11:55:29 AM UTC-4, Dave Martin
wrote:

what does expected an indented block


*what does an indented block mean?


It means that the line of code belongs to a certain body as defined
above its position.

Please follow the tutorial.

...


this is my code the error is on line 15



Did you follow the tutorial?

Is it easy (for *volunteer* helpers) to work-out which is line 15?

Back to the original question: Is the (loop) code actually indented?

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: NEWBIE: how to get text onto 2 lines on a 16x2 lcd display

[DL Neil via Python-list]

On 26/09/19 9:14 PM, RobH wrote:
I have some sample/demo python code for scrolling and outputting text 
onto a 16x2 lcd display.


I would like to put my own message or text outputting to the lcd on 2 
lines. I have tried using lcd.message('my message',1) and 
lcd.message('my message', 2), but the output says:


TypeError: message() takes exactly 2 arguments (3 given)

I have also seen this on the, stackexchange site:
lcd_string("your text " + str(yourVar), 1)

But what is str(yourVar), as I assume it means a variable.
If I could have a working example please, that would be great.



I'm wondering if "lcd_string" should be "lcd.string" (per "lcd.message") 
 - would it be better to post the actual (not) working code?


Suggest you fire-up the Python REPR:

python3 # on a Linux terminal

then:

import ***whatever the LCD package is called ***
help( ***whatever...called *** )

The output from this will tell you the names of all the entities within 
the package. Within that you will be able to check for the pertinent 
class (from which lcd was derived) and see what it says about arguments 
for the message() and/or string() methods - particularly the number of 
arguments and their data-type(s).

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: NEWBIE: how to get text onto 2 lines on a 16x2 lcd display

[DL Neil via Python-list]

On 27/09/19 7:21 AM, RobH wrote:

On 26/09/2019 17:51, Dennis Lee Bieber wrote:
On Thu, 26 Sep 2019 11:58:15 +0100, RobH  declaimed 
the

following:

...


Check out this guide for info on using character LCDs with the
CircuitPython library:
https://learn.adafruit.com/character-lcds/python-circuitpython
"""

...

As I said I am a newbie with python and I did not realise that this 
would do what I wanted, doh!



We all have to start somewhere!

As it happens, I was hoping to work on a Raspberry Pi4 project, and have 
been looking at adding cameras and displays. Sadly/ironically I will be 
working on a different project which involves an nVidia Jetson Nano - so 
I'll have to look-up CUDA (apparently it is not a car, nor is it a fish 
that I'd be happy to help you eat...)


The question I wanted to ask/suggestion to make: is there a 'development 
environment' which enables development on a 'bigger PC', for later 
delivery=download to the SBC? It might make coding and unit-testing a 
bit easier (although perhaps not when specific (display) hardware enters 
the picture).


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

pathlib

[DL Neil via Python-list]

Should pathlib reflect changes it has made to the file-system?


Sample code, below, shows pathlib identifying a data-file and then 
renaming it. Yet, after the rename operation, pathlib doesn't recognise 
its own change; whereas the file system does/proves the change was 
actioned.



$ touch before.file
$ python3
Python 3.7.4 (default, Jul  9 2019, 16:48:28)
[GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> p = pathlib.Path( "before.file" )
>>> p
PosixPath('before.file')
>>> p.name
'before.file'
>>> p.rename( "after.file" )
>>> p.name
'before.file'
>>> exit()
$ ls -lah after*
-rw-rw-r--. 1 dn dn 0 Sep 30 16:05 after.file
$ ls -lah before*
ls: cannot access 'before*': No such file or directory


I would expect pathlib to keep track of changes it has made. Any 
ideas/good reasons why/why not?

NB "name" is a r/o attribute.
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 6:13 AM, Dan Sommers wrote:

On 9/30/19 12:51 PM, Chris Angelico wrote:

On Tue, Oct 1, 2019 at 1:51 AM Dan Sommers

...


All I'm doing is defending the OP, who was surprised that
renaming a file *using a Path instance* didn't reflect that
operation *in that Path instance*.  I believe that such a
surprise is reasonable; others apparently don't.

Yes, there are a lot of reasons that it is the way it is, and a
lot of reasons not to change it now.  I get that.  If the OP is
still here, then I'd like to think that the OP also gets that.



The OP is still 'here' - has been asleep = lazy toad!

Thank you. Yes, that is a reasonable summary. Yes it did catch me by 
surprise - that I couldn't rely upon the instance to reflect 'reality' 
(IMHO).


I see there is a different point-of-view. To adjust my mind-set and 
properly appreciate the discussion, I'd like to re-read the docs with 
that in-mind, first...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 30/09/19 9:28 PM, Barry Scott wrote:

On 30 Sep 2019, at 05:40, DL Neil via Python-list  
wrote:

Should pathlib reflect changes it has made to the file-system?


I think it should not.


The term "concrete" is applied to Path(), PosixPath(), and WindowsPath() 
- whereas the others are differentiated with the prefix "Pure".


I take "concrete" to mean 'existing in reality or real experience'. 
Thus, I saw the Pure* entities as supporting abstract paths, but the 
concrete entities as representing (and reflecting) real-world (file 
system) entities.


Thus, there is no need for .exists() to be available in the Pure paths, 
but there is when utilising their concrete implementations.


NB .name however is inherited from PurePath()!



A Path() is the name of a file it is not the file itself. Why should it
track changes in the file system for the name?


BUT... Path() does keep track of changes in the file system for other 
attributes! So, why not also name?


(remember that Python (in our code examples) does not know the file by 
its file-name (before OR after rename) but by the instance name, eg 
"save_file", below)




Here is code to show why following the rename will break logic:

save_file = pathlib.Path('my-data.txt')

def save( data ):
# backup file
if save_file.exists():
save_file.rename('my-data.backup')

# save data
with save_file.open() as f:
f.write( data )

while True:
save( generate_data() )
time.sleep( interval )

If the rename of the file changed the path the above code will fail.



That is one use-case, but in the use-case which led to the original 
post, the logic was:


iterate directory-entries,
if fileNM follows an out-of-date naming-convention,
rename the file,
then process further
(logging observations along the way).

Here's a code-snippet illustrating both of the above points:

import pathlib
p = pathlib.Path( "data-file" )
p
# PosixPath('data-file')
p.stat()
os.stat_result(... st_mtime=1569898467, st_ctime=1569898467)
# ... = excised results, leaving two data-points worth observing

with p.open("w") as f:
f.write("new stuff")
# 9

p.stat()
os.stat_result(... st_mtime=1569898572, st_ctime=1569898572)
# hey look, this reflects REAL and CHANGED data from the FS

# using input logic, cf previous example's output logic
p.rename( "new-name" )
with p.open( "r" ) as f: f.readline()
...
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1193, in open
opener=self._opener)
  File "/usr/lib64/python3.7/pathlib.py", line 1046, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'data-file'

# hence we cannot follow the use-case

# why? Because .name == 'data-file' but the real-world file is (now) 
called "new-name".


NB I would not attempt to suggest that the logic of the 'write' use-case 
is any more, or any less, valid that that of the 'read' use-case; and 
I'm not arguing with you personally!



Looking at the PEP, it didn't alleviate my confusion because:
(a) <<<
Why an object-oriented API
... form of filesystem handling abstraction
>>>
(b) <<<
Immutability
Path objects are immutable, which makes them hashable and also prevents 
a class of programming errors.

>>>
(c) <<<
Concrete paths API
In addition to the operations of the pure API, concrete paths provide 
additional methods which actually access the filesystem to query or 
mutate information.

>>>

I liked the OOP aims in point (a) but it clearly says "filesystem".

Whereas the logic (mentioned above) of the inherent 'safety' of 
immutable objects is inescapable - point (b), that is incompatible with 
the usage and behavior of a file system. [which is a point made (by 
others), elsewhere)]


Point (c) appears to suggest (as written above) that whereas the Pure 
API can be immutable and separate from any physical file system, the 
Concrete paths will perform actions and thus mutate with the real FS.



Further to my (personal) point about use-cases and debate, if we follow 
the 'recommended' code snippet in the (PSL) manual, the logic is that 
there should be separate paths/Paths (whether for the two (physical) 
files or not) - their close relationship notwithstanding:


>>> p = Path('foo')
>>> p.open('w').write('some text')
9
>>> target = Path('bar')
>>> p.rename(target)
>>> target.open().read()
'some text'

NB utilising two Path instances (cf allowing a string to 'intrude') wil

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 1:09 AM, Chris Angelico wrote:

On Mon, Sep 30, 2019 at 9:54 PM Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:

I would have said the same thing, but the docs⁰ disagree:  a
PurePath represents the name of (or the path to) a file, but a
Path represents the actual file.


⁰ https://docs.python.org/3/library/pathlib.html


I don't think it represents the actual file. If it did, equality would
be defined by samefile, NOT by the file name.


from pathlib import Path
import os
open("file1", "w").close()
os.link("file1", "file2")
Path("file1") == Path("file2")

False

Path("file1").samefile(Path("file2"))

True

Path("file1") == Path("file1")

True



This example involves a "hard link" in Linux - can't recall if it 
applies under MS-Win...


On a Linux system the 'two' files share the same inode, and thus the 
same 'patch of disk', but one can be deleted without affecting the 
other. Thus, borrowing from the above snippet:


>>> import os
>>> open("file1", "w").close()
>>> os.link("file1", "file2")
>>> f1 = pathlib.Path( "file1" )
>>> f2 = pathlib.Path( "file2" )
>>> f1 == f2
False
### they are not the same path (by name) - due only to the last character

>>> f1.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=2, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569903410)

>>> f2.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=2, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569903410)

>>> f1.samefile( f2 )
True
### but they do 'label' the same file when the path is applied to the FS.

Let's try something similar, but the other-way-around:

>>> f1 = pathlib.Path( "file1" )
>>> f1.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=1, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569903851)

### this path exists in the FS

>>> f2 = pathlib.Path( "not-file1" )
>>> f2.stat()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1168, in stat
return self._accessor.stat(self)
FileNotFoundError: [Errno 2] No such file or directory: 'not-file1'
### this path does not describe a file in the FS

>>> f1.rename( f2 )
### here's a rename operation per the manual!
>>> f1.stat()
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/pathlib.py", line 1168, in stat
return self._accessor.stat(self)
FileNotFoundError: [Errno 2] No such file or directory: 'file1'
### but f1 does not represent a 'real file' any more

>>> f2.stat()
os.stat_result(st_mode=33204, st_ino=1049466, st_dev=64769, st_nlink=1, 
st_uid=1000, st_gid=1000, st_size=0, st_atime=1569903409, 
st_mtime=1569903409, st_ctime=1569904293)

### f2 is a path which represents a real-world file though
### where have we seen those numbers before?


> It still represents the path to the file, not the file itself, and if
> you move something over it, it will see the new file.

I like this description. Thanks!

That said, maybe pathlib should have stuck with paths/PurePaths, and we 
should have something else (logically separate, but closely-related) 
which manipulates the files themselves?
(preferably OO, and tidying-up/offering an alternative to the morass of 
os, os.path, sys, shutil, and ?others)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 1:40 AM, Barry Scott wrote:




On 30 Sep 2019, at 12:51, Dan Sommers <2qdxy4rzwzuui...@potatochowder.com> 
wrote:

On 9/30/19 4:28 AM, Barry Scott wrote:

On 30 Sep 2019, at 05:40, DL Neil via Python-list  
wrote:
Should pathlib reflect changes it has made to the file-system?

I think it should not.
A Path() is the name of a file it is not the file itself. Why should it
track changes in the file system for the name?


I would have said the same thing, but the docs⁰ disagree:  a
PurePath represents the name of (or the path to) a file, but a
Path represents the actual file.


I'm not seeing that wording in the python 3.7 pathlib documentation.
Can you quote the exact wording please?

I do see this:

"Pure path objects provide path-handling operations which don’t actually access a 
filesystem."

And:

"Concrete paths are subclasses of the pure path classes. In addition to 
operations provided
by the latter, they also provide methods to do system calls on path objects."

There is no requirement that a Path() names a file that exists even.



That is true - of a "path".

However, as soon as said path is applied to the file system, eg .open( 
"r" ) it had better exist or we will disappear into OSError-country.


If we use .open( "w" ) then the very real file is created within the FS, 
at that path.


So, the file does not need to exist for a path to be legal - which is a 
(to me sounds like) PurePath.


However, don't we have to suspend such 'purity' in the "concrete" classes...


That said, why doesn't your argument apply to read and write?  I
would certainly expect that writing to a path and then reading
from that same path would return the newly written data.  If I
squint funny, the Path object is tracking the operations on the
file system.


I do not expect that. Consider the time line:

1. with p.open('w') write data
2. external process changes file on disk
3. with p.open('r') read data

How would (3) get the data written at (1) guaranteed?
It will lead to bugs to assume that.

The path object is allowing system calls that need a file's path to be called,
that is all. Beyond that there is no relationship between the pathlib.Path()
objects and files.


In theory one can separate paths and files. In practice, if Python is to 
access a file it must pass a path to the OpSys/FS.


To be fair, the temporal argument (race condition) applies to any IO 
method unless "locking" is available (it is not, either in pathlib or 
the os and sys libraries.


You cannot .open( 'r' ) unless the actual file exists.

You (probably) won't want to .open( 'w' ) if the actual file exists.

Wouldn't we like a library which helps, in both cases!



I think I'm actually arguing against some long since made (and
forgotten?) design decisions that can't be changed (dare I say
fixed?) because changing them would break backwards
compatibility.


+1



Yuck.  :-)  And I can absolutely see all sorts of different
expecations not being met and having to be explained by saying
"well, that's the way it works."


I'd suggest that the design is reasonable and If there is misunderstanding that 
its
something that docs could address.


Design: +1
Misunderstanding: +1
Docs: +1

Perhaps pathlib encourages one/some to believe that it has been designed 
to do more than it does/set out to do?


Perhaps it does more than it should (by implementing some closer 
relationships with the underlying file system, but not others)?


Perhaps it makes sense for .name to be a data-attribute in the Pure 
classes, but it should be name(), ie a method-attribute in the concrete 
classes? (per other methods subject to change/mutation - see .stat() 
examples in earlier responses)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 3:21 AM, Dan Sommers wrote:

On 9/30/19 8:40 AM, Barry Scott wrote:
  >> On 30 Sep 2019, at 12:51, Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:
  >> On 9/30/19 4:28 AM, Barry Scott wrote:
  >>>> On 30 Sep 2019, at 05:40, DL Neil via Python-list
 wrote:
  >>>> Should pathlib reflect changes it has made to the file-system?
  >>> I think it should not.
  >>> A Path() is the name of a file it is not the file itself. Why 
should it

  >>> track changes in the file system for the name?
  >>
  >> I would have said the same thing, but the docs⁰ disagree:  a
  >> PurePath represents the name of (or the path to) a file, but a
  >> Path represents the actual file.
  >
  > I'm not seeing that wording in the python 3.7 pathlib documentation.
  > Can you quote the exact wording please?
  >
  > I do see this:
  >
  > "Pure path objects provide path-handling operations which don’t
actually access a filesystem."
  >
  > And:
  >
  > "Concrete paths are subclasses of the pure path classes. In addition
to operations provided
  > by the latter, they also provide methods to do system calls on path
objects."

That's the wording I read.  I inferred that "path-handling operations
which don't actually access a filesystem" meant an object that didn't
necessarily represent an actual file, and that "provide methods to do
system calls on path objects" did indicate an actual file.  From the
existence of Path.read_bytes, I inferred that at least some Path objects
represent (and operate on) actual existing files.  I've been doing this
for a long time, and I may have read my expecations into those words.


+1 "Pure" cf "concrete".

The mixture makes it difficult to insist that a Path does not represent 
a file if (some) operations are included.




  > There is no requirement that a Path() names a file that exists even.
Agreed.

  >> That said, why doesn't your argument apply to read and write?  I
  >> would certainly expect that writing to a path and then reading
  >> from that same path would return the newly written data.  If I
  >> squint funny, the Path object is tracking the operations on the
  >> file system.
  >
  > I do not expect that. Consider the time line:
  >
  > 1. with p.open('w') write data
  > 2. external process changes file on disk
  > 3. with p.open('r') read data
  >
  > How would (3) get the data written at (1) guaranteed?
  > It will lead to bugs to assume that.

I didn't say anything about a guarantee, or about an external processes.
If I have a single process that writes data to a file and then reads
from that file, I would expect to read what I just wrote.  See the
documentation of Path.read_bytes and Path.write_bytes.  If I throw an
external process, or a networked file system, or multiple layers of
buffering and/or caching into the mix, then all such bets are off.

I think you're making my point about expectations.  :-)


+1



  > The path object is allowing system calls that need a file's path to
be called,
  > that is all. Beyond that there is no relationship between the
pathlib.Path()
  > objects and files.

The documentation of Path.read_bytes and Path.write_bytes say otherwise.


+1



  >> I think I'm actually arguing against some long since made (and
  >> forgotten?) design decisions that can't be changed (dare I say
  >> fixed?) because changing them would break backwards
  >> compatibility.
  >>
  >> Yuck.  :-)  And I can absolutely see all sorts of different
  >> expecations not being met and having to be explained by saying
  >> "well, that's the way it works."
  >
  > I'd suggest that the design is reasonable and If there is
misunderstanding that its
  > something that docs could address.

I'm not disagreeing.  I suspect that we've both worked on enough
different systems to know that not all OSes, file systems, libraries,
and versions and combinations thereof work the same way under all
circumstances (multiple threads, multiple processes, caching, buffering,
etc.).  It's the epitome of YMMV.

Rename is a particularly thorny case because renaming a file, at least
on a POSIX system, is an operation on the directory containing the file
rather than the file itself.



Thank you @Dan for keeping the conversation going during my night-hours.
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 1/10/19 2:58 PM, Dan Sommers wrote:

On 9/30/19 3:56 PM, Barry Scott wrote:
On 30 Sep 2019, at 16:49, Dan Sommers 
<2qdxy4rzwzuui...@potatochowder.com 
> wrote:



In the totality of a Path object that claims to represent paths
to files,


It represents string that *should* in most cases work in the APIs
that work on files. Not the same idea.


I'm not sure the documentation supports your view.  Components
of paths are strings, but IMO the Path object represents a file.


Worse! There may be an argument for suggesting that the documentation 
supports both descriptions...




including the arguably troublesome read_bytes and
write_bytes methods, and a rename method, however, it's not
unreasonable expect the object to track *itself* when I use *its*
own rename method (backwards compatibility restraints
notwithstanding).


"the object" did track the changes its just that "the object" is not
the Path object, it's in the operating system and/or the file system.
For the rename it's the directory that the name is recorded in.


So which is it?  Does the Path object represent a string, the
name of a file (whether that file exists or not), or the file
itself?  A moment ago, you claimed that a Path object represents
a string that should work in the APIs that work on files.  Now,
you're claiming that the Path object is a proxy for something in
the filesystem.  I don't mean to be combative or confrontational,
but I think that this fuzziness/inconsistency is at or near the
root of the differing expectations.


Yes, see earlier comment about "confusion" - in both the minds of 
learners/newcomers and those who are 'experienced'.




There was an interest talk at this years PYCON UK about the
the errors that people new to python make. Misunderstand syntax
is about 1/3 of the problems, but 2/3 was having the wrong model.

This discussion seems to fall into the "wrong model" world that
leads to a expectation failure.


On this (that there's something about the model of Path objects
that leads to expectation failures) we agree.  :-)


Are we talking about "Here's Your Mistake" 
(https://www.youtube.com/watch?v=7gMOaWdzDSw)?


You've strayed into my research area: cognitive psychology - 
understanding how people learn. Whilst we do indeed talk about "models" 
- a person's understanding of how things work and/or how they fit together.


"Models" in the computing world are more often called "abstractions". 
Every?most computer program(mes) are, by definition, an abstraction of 
some real-world event, calculation, or whatever. A Person() class might 
have the name of a real-person, but it is definitely not that (or any 
other) person. It is an abstraction (and likely only of 'part' of that 
person). That said, if the person changes his/her name by Deed Poll 
(legal process) to "Slinky Python", then it would be insulting to use 
the previous name(s)! (and if someone has divorced and re-married, 
never, never, NEVER use the names of the original him+her when referring 
to the 'new' him+her!!!)

(PC alert: traditional pairing described)

You will observe that the original post attempts to paint a picture of 
the model I expected. Since then, several folk have taken me up on the 
request for "good reasons ... why not", and thus, as you say, enabled 
the replacement (education) of my "wrong model", with "the real thing".


From where do people acquire their models? The YouTube video briefly 
touched-on this. Most commonly: previous experience - thus math students 
write formulae as description, and can come to grief when Python code 
insists upon a sequence of instructions.


So, next time someone tells you to 'read the code' you will wonder why 
the built-in 'documentation' isn't up-to-snuff; and when someone says 
RTFM you'll also realise that you've invited the person to construct 
their own model - an error-prone process at best; and that is why, if 
one examines the rate of adoption for (open source) languages and 
libraries, the greatest influence is the presence/absence of quality 
tutorials - materials designed to encourage the building of congruent 
mental models!



In this discussion we've heard people comparing pathlib with previous 
facilities in os and sys. That's unfortunate because reading the PEP 
shows an intention to diverge.


At the same time, and again reviewing this discussion, you can observe 
two models: one more abstract ("Pure") and the other more oriented to 
reflecting the FS ("concrete"). The gap between is the importance one 
places on each of the two models. Favoring the abstract means seeing the 
Pure constructs as important and the concrete as lesser. The more 
practical view asking for a means to access the file system, sees the 
Pure model as a foundation.



Have we moved on to how we can improve the situation?


Absolutely.

Documenting the fact that calling rename on a Path object does
not update that object, and at least an acknowledgement of the
b

Re: pathlib

[DL Neil via Python-list]

On 2/10/19 12:52 AM, Rhodri James wrote:

On 01/10/2019 06:03, DL Neil via Python-list wrote:

On 30/09/19 9:28 PM, Barry Scott wrote:
On 30 Sep 2019, at 05:40, DL Neil via Python-list 
 wrote:


Should pathlib reflect changes it has made to the file-system?


I think it should not.


The term "concrete" is applied to Path(), PosixPath(), and 
WindowsPath() - whereas the others are differentiated with the prefix 
"Pure".


I take "concrete" to mean 'existing in reality or real experience'. 
Thus, I saw the Pure* entities as supporting abstract paths, but the 
concrete entities as representing (and reflecting) real-world (file 
system) entities.


Thus, there is no need for .exists() to be available in the Pure 
paths, but there is when utilising their concrete implementations.


Sorry but your logic is inconsistent here.  For starters, it's not that 
there's no need for .exists() in Pure paths, it's that .exists() is 
meaningless.  Pure paths aren't related to any actual filing system (to 
paraphrase you), so existence isn't an option.


However if you insist that "concrete" means "existing in reality", then 
.exists() is unnecessary because by your very definition the path must 
exist.  The very act of creating the Path object would create the 
corresponding file or directory.  So either pathlib does something 
horrific, or your definition is wrong.



Very good! Yes, I'd picked-on .exists() purely (hah!) because it does 
not appear in PurePaths, but does in concrete Paths.


That said, it is one of the ways that a path can be shown to transition 
from some 'pure' state to become 'concrete'.


However, A.N.Other has suggested that I might be mis-applying the word 
"concrete", so maybe not. On which topic, I went looking for a decent 
technical definition of the word, but instead of coming-out smiling, 
I've been left somewhat stony-faced (hah, hah!).


A definition/description would be useful. Any pointers?


> Consider for a moment:
>
>  rhodri@scrote:~$ cat /home/rhodri/foo.txt
>  cat: /home/rhodri/foo.txt: No such file or directory
>
> cat "concatenates files" to quote its man page.  Does that mean it
> creates them if they don't exist, just because I typed the name into the
> command?  No.  I wouldn't expect it either.  In exactly the same way I
> don't expect a concrete file*name* to necessarily refer to an actual 
file.



I haven't followed this point: throwing a non-existent file at cat 
produces an err.msg. Neither of us are surprised! However, we know 
exactly which path was/paths were used, and where (at least the first) 
error was found.



In the case that sparked this enquiry, and in most others, there is no 
need for a path that doesn't actually lead somewhere. The paths that are 
used, identify files, open them, rename them, create directories, etc. 
The idea of a path that just 'exists' (sorry!), without being part of 
some purpose, seems odd.



At this time (and assuming that after two (separate) incidents dragging 
me away to solve other people's problems, I intend to stick with trying 
to get my head around pathlib - even if I have to sub-class it (which my 
reading shows is another 'can of worms'). So, 'reading' is about all 
I've accomplished since the original post. Sadly, the majority of posts 
seem to have come from other confused-minds - many of whom seemed to be 
giving-up in disgust. If true, such represents TWO failures! I'm sure 
that the designer(s) had a clear vision (having watched previous 
attempts rise-and-fall), but per earlier in this discussion, maybe the 
explanation and 'vision' could be better communicated to us simple-boys?



Regarding this discussion: I've been able to take points-raised and 
learn, even to decide on some tactics for future use. Thanks to all who 
have contributed wisdom...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 3/10/19 6:25 AM, Barry Scott wrote:
On 2 Oct 2019, at 09:14, DL Neil via Python-list 
mailto:python-list@python.org>> wrote:

On 2/10/19 12:52 AM, Rhodri James wrote:

On 01/10/2019 06:03, DL Neil via Python-list wrote:

On 30/09/19 9:28 PM, Barry Scott wrote:
On 30 Sep 2019, at 05:40, DL Neil via Python-list 
mailto:python-list@python.org>> wrote:


Should pathlib reflect changes it has made to the file-system?


I think it should not.


The term "concrete" is applied to Path(), PosixPath(), and 
WindowsPath() - whereas the others are differentiated with the 
prefix "Pure".


I take "concrete" to mean 'existing in reality or real experience'. 
Thus, I saw the Pure* entities as supporting abstract paths, but the 
concrete entities as representing (and reflecting) real-world (file 
system) entities.


Thus, there is no need for .exists() to be available in the Pure 
paths, but there is when utilising their concrete implementations.
Sorry but your logic is inconsistent here.  For starters, it's not 
that there's no need for .exists() in Pure paths, it's that .exists() 
is meaningless.  Pure paths aren't related to any actual filing 
system (to paraphrase you), so existence isn't an option.
However if you insist that "concrete" means "existing in reality", 
then .exists() is unnecessary because by your very definition the 
path must exist.  The very act of creating the Path object would 
create the corresponding file or directory.  So either pathlib does 
something horrific, or your definition is wrong.



Very good! Yes, I'd picked-on .exists() purely (hah!) because it does 
not appear in PurePaths, but does in concrete Paths.





On which topic, I went looking for a decent technical definition of 
the word, but instead of coming-out smiling, I've been left somewhat 
stony-faced (hah, hah!).


"concrete

  *
adj.
Of or relating to an actual, specific thing or instance; particular.
"


That said, it is one of the ways that a path can be shown to 
transition from some 'pure' state to become 'concrete'.


However, A.N.Other has suggested that I might be mis-applying the word 
"concrete", so maybe not.


Concrete means a specific operating system's filesystem rules, Windows 
or Posix.


At last the gears have stopped grinding! (cement dust?)

The distinction of "pure", as distinct from "concrete", was readily grasped.

The idea that "concrete" refers to FS rules, cf files on the FS, has 
been what so many have been battling to lodge between my ears. Well done!


Also, nicely stated.

In view of my not being the only one to arrive with similar 
expectations, do you/we feel that this is carefully and sufficiently 
conveyed within the (PSL) documentation?

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 3/10/19 3:07 AM, Rhodri James wrote:

On 02/10/2019 09:14, DL Neil via Python-list wrote:
That said, it is one of the ways that a path can be shown to 
transition from some 'pure' state to become 'concrete'.


However, A.N.Other has suggested that I might be mis-applying the word 
"concrete", so maybe not. On which topic, I went looking for a decent 
technical definition of the word, but instead of coming-out smiling, 
I've been left somewhat stony-faced (hah, hah!).


A definition/description would be useful. Any pointers?


I think we're looking at a philosophical split, so I'd look in that 
direction rather than for technical terminology.


My rough and ready definition *in this instance* relies on observing 
that we are supposed to contrast "pure" and "concrete" and going from 
there.


The overriding thing for me is that paths are names.  Just names.  They 
have a particular syntax, but that's all.  This is obviously true for 
pure paths, which are clearly abstractions. 
PurePath("/home/rhodri/foo.txt") cannot refer to a real file because it 
has no mechanisms for relating to reality.  It can only be a name, and 
all the PurePath class gives us is convenient mechanisms for 
manipulating that name within its syntactic rules.


Concrete paths are not pure paths.  Literally, in logic terms.  Pure 
paths cannot refer to real file, concrete paths can refer to real files. 
  They don't necessarily do so otherwise we have a massive excluded 
middle.  Path("/home/rhodri/foo.txt") may or may not actually exist on 
any computer.  It may refer to a file, and by the end of this sentence 
it may refer to a different file to what it was at the start.  The only 
sensible interpretation I can see is that it is still a name, just one 
that may transiently be related to a real object.


Concrete may not be the best term for this, but I can't think of a 
better one.



Nor I, but had assumed (having seen it before) that it was a 
generally-accepted term in OOP that I have yet to learn. Was slightly 
surprised not to find it in any of the/my usual tech.dictionaries.


Obviously, my assumptions/expectations of its meaning were inaccurate or 
incomplete, but I appreciate your efforts to straighten-out it (and me)!

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: pathlib

[DL Neil via Python-list]

On 3/10/19 12:42 AM, Dan Sommers wrote:

On 10/2/19 4:14 AM, DL Neil via Python-list wrote:


In the case that sparked this enquiry, and in most others, there is no
need for a path that doesn't actually lead somewhere. The paths that are
used, identify files, open them, rename them, create directories, etc.
The idea of a path that just 'exists' (sorry!), without being part of
some purpose, seems odd.


Think about preferences.  Users can put preferences in (e.g.)
~/.config/prefs, admins might provide "local" preferences in
/etc/prefs, and the developers might provide fallback preferences
in /etc/prefs.

When the application starts up, it could have a list of paths to files
that may or may not exist.


This is an excellent example. Without thinking I would have left such 
as-is. However, what I'm taking-in, is that in order to gain advantage 
from the semantic meaning inherent in the Path class(es), I shouldn't 
leave these as strings (the way they arrive from (current) config 
files*) but they should be defined as (Pure) Paths in the same way that 
numbers will probably be converted from string inputs. As you say, 
because system-defaults may be over-ridden by user-prefs, there's no 
point in 'proving' that such a file exists - such can wait until we 
actually issue the .open()


That said, surely we would still use a 'concrete' class, in case?


* in the case of YAML files, might we even be able to define those 
values as Path()-s...




Or think about the shell in that failed "cat" command.  It's
possible that cat created a path from what the user typed and
then tried to open it.  For that brief moment, cat had a path
to a file that didn't exist, however un-useful it may have been.

At this time (and assuming that after two (separate) incidents dragging
me away to solve other people's problems, I intend to stick with trying
to get my head around pathlib - even if I have to sub-class it (which my
reading shows is another 'can of worms'). So, 'reading' is about all
I've accomplished since the original post. Sadly, the majority of posts
seem to have come from other confused-minds - many of whom seemed to be
giving-up in disgust. If true, such represents TWO failures! I'm sure
that the designer(s) had a clear vision (having watched previous
attempts rise-and-fall), but per earlier in this discussion, maybe the
explanation and 'vision' could be better communicated to us simple-boys?


I don't think anyone gave up in disgust.  Yes, there was some


Late at night: I used the word "posts" twice, to describe two quite 
different communications. Apologies


The subject of that comment was the (other) research/reading I've been 
doing. No-one on THIS list has given the impression of wanting to dump 
pathlib (which encourages my persisting).


Certainly, although some may have quietly given-up talking to a non-OOP 
native - and one so 'slow', I am appreciative of all help-given!




disagreement, and now the discussion has slowed or stopped, but  > think your 
original question was answered:  Path objects,
apparently by an arguably questionable design, fail to meet your
expecation, and some simple changes end up breaking backwards
compatibility.  Maybe a documentation change could prevent others
from experiencing the same expectation failure.


As discussed previously, and elsewhere (just now).



Maybe you could implement one of the proposed changes in a private
library function as a workaround?


In my mind, I'm wondering if it will come to that (having 'got past' the 
original observation/issue, I'm concerned by .rename()'s silent errors, 
for example). However, that 'outside' research, eg StackOverflow, shows 
that sub-classing pathlib is problematic, and quite possibly not part of 
the design (this is repeating 'gossip' - I'm not going to try to justify 
the comment or the claim). That said, last night my code sub-classing 
Path() seemed to work quite happily (albeit only tested on a 'Posix' 
box). The yawning chasm/gaping jaws below, however, are that I've 
probably made yet another 'assumption' about how things 'should' work. 
Run for the hills!


This was supposed to be a simple, down-time task; a learning-opportunity 
re-factoring code to use a new (er, um, as of v3.4) library...


Thanks again!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Formatting floating point

[DL Neil via Python-list]

On 8/10/19 4:04 AM, boffi wrote:

DL Neil  writes:


Agreed: there's ton(ne)s of information 'out there', much of it old,
eg Python2, "formatter" (deprecated since v3.4)

?
are you referring to the `string.Formatter`[*] class?
 $ python
 Python 3.7.4 (default, Aug 13 2019, 20:35:49)
 [GCC 7.3.0] :: Anaconda, Inc. on linux
 Type "help", "copyright", "credits" or "license" for more information.
 >>> from string import Formatter
if not, what else?
g
[*] https://docs.python.org/2/library/string.html#string.Formatter
 https://docs.python.org/3.7/library/string.html#string.Formatter



As mentioned, there is a wealth of information, but I must admit it took 
a moment-or-two before I re-located the 'official' reference to f-strings:


2.4.3. Formatted string literals 
https://docs.python.org/3/reference/lexical_analysis.html#f-strings

(and please 'read-around' this section for more valuable data!)

FYI the original PEP is at https://www.python.org/dev/peps/pep-0498/


It is worth reading about string formatters (per (your) ref, above), 
because many elements of the 'Formatting mini-language' are also 
available within f-strings!



As you have probably observed, there are now (v3.6+) several methods 
which can be applied to the formatting of data. Each has its advantages, 
and whilst I have a preference for, and thus recommended, WRITING code 
to use f-strings, it is also (likely to be) important that we READ and 
comprehend the older/alternatives!



NB politesse suggests that I should apologise, but I no-longer wish to 
work with Python2; hence only mentioning Py3 'here'.

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to handle '-' in the 'from' part in a "from import" statement?

[DL Neil via Python-list]

On 8/10/19 3:45 PM, jf...@ms4.hinet.net wrote:

For example:
from my-dir import test

I know it can be solved by renaming, but any alternative?



The manual is your friend:
- import
- importlib

(the latter allows modules to be identified by strings)

However, Pythons has naming rules. If you try to get around them, sooner 
or later you'll 'forget' and trip yourself up. Recommend your first idea!

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to handle '-' in the 'from' part in a "from import" statement?

[DL Neil via Python-list]

On 9/10/19 2:12 PM, jf...@ms4.hinet.net wrote:

dieter於 2019年10月8日星期二 UTC+8下午1時33分20秒寫道：

jf...@ms4.hinet.net writes:

...
But most of the download from Github has a directory named '-master' which 
causes a trouble sometimes.


Those are likely not meant to be imported directly.

Typically, you have a "setup" step which installs (in some way)
a "distribution". This step usually ensures that you can use
"normal" Python import syntax to access all associated packages.

The "setup" step is typically performed with
"python setup.py develop" or "python setup.py install" --
with the "distribution" providing the "setup.py".


Yes, I understand the normal procedure on using a package, but it's not always 
the case.


I'm curious - if this is the correct way to do things, and the original 
author intends the package/module for you to download, why does (s)he 
choose to follow an non-Pythonic naming convention?


Does this say something about the author?
- something about his/her abilities in Python?
- the Python-friendliness (or otherwise) of GitHub? (cf PyPi)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to handle '-' in the 'from' part in a "from import" statement?

[DL Neil via Python-list]

On 9/10/19 2:46 PM, Chris Angelico wrote:

On Wed, Oct 9, 2019 at 12:36 PM DL Neil via Python-list
 wrote:

...


(Or just using pip to install directly from GitHub, although not
everyone knows that that's possible.)


Come on, you just knew I was going to ask how...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to handle '-' in the 'from' part in a "from import" statement?

[DL Neil via Python-list]

On 9/10/19 4:34 PM, jf...@ms4.hinet.net wrote:

jf...@ms4.hinet.net於 2019年10月8日星期二 UTC+8上午10時45分36秒寫道：

For example:
from my-dir import test

I know it can be solved by renaming, but any alternative?

--Jach


Maybe another (better?) solution is:
import sys
sys.path.append(r'my-dir')
import test



Another option might be to add a symlink* from the application's 
directory to wherever you've downloaded and expanded the GitHub .zip 
archive.


* Linux terminology, other OpSys may use different.

Both are somewhat unattractive (IMHO).
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Instantiating sub-class from super

[DL Neil via Python-list]

Is there a technique or pattern for taking a (partially-) populated 
instance of a class, and re-creating it as an instance of one of its 
sub-classes?



In a medically-oriented situation, we have a Person() class, and start 
collecting information within an instance (person = Person(), etc).


During the data-collection process the person's sex may become obvious, 
eg few males have become/been pregnant.


We could stick with Person() and implement specific methods therein, 
rather than separate Man and Woman sub-classes, but...


It seemed better (at the design-level) to have Man( Person ) and Woman( 
Person ) sub-classes to contain the pertinent attributes, source more 
detailed and specific questions, and collect such data; by gender.


In coding-practice, once gender becomes apparent, how should the 
instance of the Man() or Woman() sub-class be created - and established 
with the ID and other attributes previously collected as a Person instance?


This attempt seems hack-y:

man = Man()
man.__dict__.update( person.__dict__ )


Is there a pythonic tool for such, or is the task outlined 
fundamentally-inappropriate?


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

Hi Greg,


On 15/10/19 11:37 AM, Gregory Ewing wrote:

DL Neil wrote:
Is there a technique or pattern for taking a (partially-) populated 
instance of a class, and re-creating it as an instance of one of its 
sub-classes?


Often you can assign to the __class__ attribute of an instance
to change its class.

Python 3.7.3 (default, Apr  8 2019, 22:20:19)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> class Person:
...  pass
...
 >>> class Male(Person):
...  pass
...
 >>> p = Person()
 >>> p.__class__ = Male
 >>> isinstance(p, Male)
True
 >>>


Brilliantly easy. Thanks!

Is this manipulation documented/explained anywhere? Would you describe 
it as 'good practice', or even: sensible?




You would then be responsible for initialising any attributes of
Male that Person didn't have.


Understood.

The contents of Person.__init__( -whatever- ) are executed at instantiation.

The contents of Male.__init__( - whatever- ) will be executed during a 
'normal' instantiation.


If, however, a Person-instance is 'converted'* into a Male-instance, 
then Male.__init__() will not be executed.

(haven't bothered to experiment with an explicit call, because...)


In this case, that is not an issue, because apart from the value of 
"sex", the __init__() actions are all provided by super().


Further experimentation revealed something I wasn't sure to expect. Thus 
although:


class Male( Person ):
etc
class Person():
etc

won't work, because Person has yet to be defined; in the correct 
sequence, we *can* 'anticipate' names within the super-class:


class Person():
etc
def convert( self ):
self.__class__ = Male
class Male( Person ):
etc

Agreed, better is:

def convert( self, sub_class ):
self.__class__ = sub_class
...
p = Person()
p.convert( Male )

Amusingly enough, could "convert"* the p-instance again-and-again.

* careful terminology!


PoC code:

class Person():

def __init__( self ):
self.ID = "Respondent"
self.converter = { "M":Male, "F":Female }


def questionnaire( self, sex ):
self.sex = sex

def super_function( self ):
print( "Pulling on tights and donning cape..." )

def convert( self ):
self.__class__ = self.converter[ self.sex ] 


class Male( Person ):

def male_only( self ):
print( "I am secure in my man-hood" )


class Female( Person ):

def female_only( self ):
print( "Thy name is vanity" )


p = Person()

print( "PersonOBJ ~", p )
print( "Person __class__ ~", p.__class__ )
print( "PersonID ~", p.ID )

p.questionnaire( "M" )
print( "M, MaleOBJ ~", p.sex, p.converter[ p.sex ] )
p.convert()

print( "PersonOBJ ~", p )
print( "Person __class__ ~", p.__class__ )
print( "PersonID ~", p.ID )

p.male_only()
p.super_function()

p.questionnaire( "F" )
print( "F, FemaleOBJ ~", p.sex, p.converter[ p.sex ] )
p.convert()

print( "PersonOBJ ~", p )
print( "Person __class__ ~", p.__class__ )
print( "PersonID ~", p.ID )

p.female_only()
p.super_function()

p.male_only()   # Exception


Output:
Showing that:
- the object remains the same, even as its type/class is 'converted'
- the object retains any value established before 'conversion'
or, the 'conversion' process carries-across all data-attributes
- after conversion the sub-class's methods become available
- after conversion the super-class's methods remain available
- after a further 'conversion', the same experience
but methods particular to the first conversion have been 'lost'.
(as expected)

[~]$ python3 person.py
PersonOBJ ~ <__main__.Person object at 0x7f014085bdd0>
Person __class__ ~ 
PersonID ~ Respondent
M, MaleOBJ ~ M 
PersonOBJ ~ <__main__.Male object at 0x7f014085bdd0>
Person __class__ ~ 
PersonID ~ Respondent
I am secure in my man-hood
Pulling on tights and donning cape...
F, FemaleOBJ ~ F 
PersonOBJ ~ <__main__.Female object at 0x7f014085bdd0>
Person __class__ ~ 
PersonID ~ Respondent
Thy name is vanity
Pulling on tights and donning cape...
Traceback (most recent call last):
  File "person.py", line 58, in 
p.male_only()   # Exception
AttributeError: 'Female' object has no attribute 'male_only'


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

On 16/10/19 12:38 AM, Rhodri James wrote:

On 14/10/2019 21:55, DL Neil via Python-list wrote:

...

It seemed better (at the design-level) to have Man( Person ) and 
Woman( Person ) sub-classes to contain the pertinent attributes, 
source more detailed and specific questions, and collect such data; by 
gender.


Knowing a lot of Trans people as I do, may I gently suggest that this 
solution will find many and varied ways of coming back to bite you?



[with more respect than the previous humor]


You are absolutely correct. The use-case was (over-)simplified, per 
list-advice.


That said, if a "trans" person has ovaries or testes (for example) then 
a non-traditional sexual identification is irrelevant - for medical 
purposes. Diseases in those areas (and now I'm a long way from a 
research questionnaire and from Python - but this is roughly how it was 
explained to me) still apply, and sadly, may in-fact be considerably 
complicated by any medical processes that may have contributed to a 
transition.


So, yes, the "label" is unimportant - except to politicians and 
statisticians, who want precise answers from vague collections of 
data... (sigh!)


FYI: This country has been leading the way, to the point where even 
asking such questions is no longer allowed under many circumstances.


Back to Python: yes, the model is considerably complicated because there 
are no 'straight lines' to divide - that, and the rather arcane 
DB-structure we've inherited (which contributed to pilot-ing the 
sub-class route) are leading us back to the idea of a 'monolithic' 
Person class* with loads of data-points/flags and 
conditional-executions, to take care of individual differences and 
meeting the (medical) objectives of the questionnaire. Perhaps one of 
the physicians will prescribe a head-ache remedy?


* without denigrating the generosity of those who helped with the OP

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

On 16/10/19 1:55 PM, duncan smith wrote:

On 15/10/2019 21:36, DL Neil wrote:

On 16/10/19 12:38 AM, Rhodri James wrote:

On 14/10/2019 21:55, DL Neil via Python-list wrote:

...
So, yes, the "label" is unimportant - except to politicians and
statisticians, who want precise answers from vague collections of
data... (sigh!)



[snip]

No not (real) statisticians. People often want us to provide precise
answers, but they don't often get them.

"It ain’t what you don’t know that gets you into trouble. It’s what you
know for sure that just ain’t so." (Mark Twain - perhaps)


+1

Although, you've undoubtedly heard people attempt to make claims of 
having 'accurate figures' (even, "that came from Stats") when you told 
them that the limitations and variations rendered the exercise laughable...


My favorite (of the moment) is a local computer store who regularly 
offer such gems as: (underneath the sales (web-) page for an upmarket 
*desktop* computer)  "people who bought this also bought" followed by at 
least two portable PC carry cases. They must be rather large carry-bags! 
(along with such surprises as keyboard, mouse, ...)


This morning I turned-down a study for a political group. One study has 
already been completed and presented. The antagonist wanted an A/B 
comparison (backing his 'side', of course). I mildly suggested that I 
would do it, if he'd also pay me to do an A/B/C study, where 'C' was a 
costing - the economic opportunity cost of 'the people' waiting for 'the 
government' to make a decision - (and delaying that decision by waiting 
for "study" after "study" - The UK and their (MPs') inability to decide 
"Brexit" a particularly disastrous illustration of such)



Sorry, don't want to incur the anger of the list-gods - such 
calculations would be performed in Python (of course)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

On 16/10/19 6:33 PM, Frank Millman wrote:

On 2019-10-14 10:55 PM, DL Neil via Python-list wrote:
Is there a technique or pattern for taking a (partially-) populated 
instance of a class, and re-creating it as an instance of one of its 
sub-classes?


Here is a link to an article entitled 'Understanding Hidden Subtypes'. 
It dates back to 2004, but I think it is still relevant. It addresses 
precisely the issues that you raise, but from a data-modelling 
perspective, not a programming one.


http://tdan.com/understanding-hidden-subtypes/5193

I found it invaluable, and applied the concepts in my own 
business/accounting application. Having created the ability to make 
subtypes visible and explicit, I found all kinds of unexpected uses for 
them.


The article seems to be missing a couple of images (Figure 1 and Figure 
2) showing the data relationships. I downloaded the original article 
onto my computer years ago, and my local copy does have the images, so 
if you would like to see them let me know and I will upload my version 
somewhere to make it accessible.


Superb!

Yes please Frank - I've also approached it from the data/DB side, and 
thus presumably why I was puzzling over how one implements in Python.


(alternatively, email a PDF/similar directly)
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

On 17/10/19 4:08 AM, Piet van Oostrum wrote:

DL Neil  writes:


That said, if a "trans" person has ovaries or testes (for example) then
a non-traditional sexual identification is irrelevant - for medical
purposes. Diseases in those areas (and now I'm a long way from a
research questionnaire and from Python - but this is roughly how it was
explained to me) still apply, and sadly, may in-fact be considerably
complicated by any medical processes that may have contributed to a
transition.


So what if a person has both ovaries and testes?
It is better to keep all options open rather than making hasty subdivisions.


Exactly!

The objective is not to sub-divide, but to ensure that those to whom 
various questions are relevant have their answers recorded, but those 
for whom the section is irrelevant are not over-burdened.




In this context that means attributes (that can be None) rather than subclasses.


Which seems to be the way we're headed...

Thanks!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

On 17/10/19 7:52 AM, MRAB wrote:

On 2019-10-16 19:43, duncan smith wrote:

On 16/10/2019 04:41, DL Neil wrote:

On 16/10/19 1:55 PM, duncan smith wrote:

On 15/10/2019 21:36, DL Neil wrote:

On 16/10/19 12:38 AM, Rhodri James wrote:

On 14/10/2019 21:55, DL Neil via Python-list wrote:

...
So, yes, the "label" is unimportant - except to politicians and
statisticians, who want precise answers from vague collections of
data... (sigh!)



[snip]

No not (real) statisticians. People often want us to provide precise
answers, but they don't often get them.

"It ain’t what you don’t know that gets you into trouble. It’s what you
know for sure that just ain’t so." (Mark Twain - perhaps)


+1

Although, you've undoubtedly heard people attempt to make claims of
having 'accurate figures' (even, "that came from Stats") when you told
them that the limitations and variations rendered the exercise 
laughable...


My favorite (of the moment) is a local computer store who regularly
offer such gems as: (underneath the sales (web-) page for an upmarket
*desktop* computer)  "people who bought this also bought" followed by at
least two portable PC carry cases. They must be rather large carry-bags!
(along with such surprises as keyboard, mouse, ...)

This morning I turned-down a study for a political group. One study has
already been completed and presented. The antagonist wanted an A/B
comparison (backing his 'side', of course). I mildly suggested that I
would do it, if he'd also pay me to do an A/B/C study, where 'C' was a
costing - the economic opportunity cost of 'the people' waiting for 'the
government' to make a decision - (and delaying that decision by waiting
for "study" after "study" - The UK and their (MPs') inability to decide
"Brexit" a particularly disastrous illustration of such)


Sorry, don't want to incur the anger of the list-gods - such
calculations would be performed in Python (of course)


Clearly, all such analyses should be done in Python. Thank God for rpy2,
otherwise I'd have to write R code. It's bad enough having to read it
occasionally to figure out what's going on under the hood (I like
everything about R - except the syntax).
 > I have too many examples of people ignoring random variation, testing
hypotheses on the data that generated the hypotheses, shifting the
goalposts, using cum / post hoc ergo propter hoc reasoning, assuming
monocausality etc. In some areas these things have become almost
standard practice (and they don't really hinder publication as long as
they are even moderately well hidden). Of course, it's often about
policy promotion, and the economic analyses can be just as bad (e.g.
comparing the negative impacts of a policy on the individual with the
positive impacts aggregated over a very large population). And if it's
about policy promotion a press release is inevitable. So we just need to
survey the news media for specific examples. Unfortunately there's no
reliable service for telling us what's crap and what isn't. (Go on,
somebody pay me, all my data processing / re-analysis will be in Python
;-).)


Even when using Python, you have to be careful:

Researchers find bug in Python script may have affected hundreds of studies
https://arstechnica.com/information-technology/2019/10/chemists-discover-cross-platform-python-scripts-not-so-cross-platform/ 



I think both of our 'Python' comments were made tongue-in-cheek. Sadly 
the tool won't guarantee the result...



At my first research project, before I'd even completed my first degree, 
I noticed a similar fault in some code*. There was I, the youngest, 
newest, least-est member of staff, telling the prof/boss and all the 
other researchers that they'd made a serious error, upon which various 
papers had been based plus a white-paper for government consideration. Oops!


(Basic-Plus on DEC PDP/Vax-en introduced a 'virtual storage array', ie 
on-disk cf in-RAM. However, it did not wipe the disk-space prior to use 
(whereas arrays were zero-ed, IIRC). Thus, random data purporting to be 
valid data-entered. Once corrected and re-run "my results" (as they were 
termed - not sure if insult or compliment) were not hugely different 
from the originals).


All we can do, is add some checks-and-balances rather than relying on 
'the computer'.


Upon which point: those of us who learned 'complicated math' with the 
aid of a slide-rule, employ a technique of mentally estimating the 
result in both the first first few digits and scale - and thus noticing 
any completely incongruous 'result'. Even with lessons in "The 
Scientific Approach" am not aware that the 'calculator' or 'computer 
generations' were/are taught such 'common sense'...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

On 18/10/19 9:27 AM, Eryk Sun wrote:

On 10/17/19, MRAB  wrote:

On 2019-10-17 20:06, Eryk Sun wrote:


I'm bugged by how the article mis-characterizes the fundamental
problem. The operating system has nothing to do with the order of a
directory listing, which varies even with an OS, depending on the file
system. The latter could store each entry in a tree structure that's
sorted by filename, or it could use a mapping with hash values, or it
could simply use the first available slot in a list, based on the
order of past create and delete operations.


"The operating system has nothing to do with the order of a directory
listing"?

Well, that depends on the operating system. It might guarantee the order.


That would be typically provide no benefit and be an unnecessary
expense in the kernel, especially for directories that contain
thousands of entries, considering applications typically need a
particular sort order anyway. Also, while such a system could make
this default order cheap for its own platform-restricted file systems,
supporting other file systems such as NTFS and ext3 would bear the
cost of having to dynamically sort directory listings instead of just
listing the entries as they're naturally stored in the file-system
data structures.



To be fair to the original developers, not many people develop with 
multiple platforms in-mind. Perhaps more *nix devs do think 'platform' 
because the user-community may be MS-Win-based...


When building an application, if it works for you, that's great. If 
others then want to use it, there is an amount of "caveat emptor" - 
particularly when jumping platforms.


With the gift of hind-sight, some ideas may seem weak, even 
reprehensible; but they worked at the time. Just this morning I was 
reading about 'legal' Linux file/path syntax, and saw an article which 
recounted that Bourne (as in -Shell) had a series of files - one for 
each of the legal characters, which could be used as some sort of 
lookup. Are there easier ways once RAM has become cheap(er)?

NB have no idea of the veracity of this claim.

I've been working on a consolidation of personal file/dir-manipulation 
utilities (as many 'here' know, and have kindly guided/assisted) but 
have not given any thought to non-Posix systems at all (contrary to the 
aims of importlib, for example). If someone takes *my* code and tries to 
run it on MS-Win Python, they'll soon find-out! That said, I'm not 
expecting to publish any results with the expectation of world-domination...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Black

[DL Neil via Python-list]

Top posting?


Agreed. As my eyes age (they're even older than my teeth!) I find the 
additional horizontal white space improves (my) comprehension, 
particularly when dealing with a dense nesting of structures.


Of course the more 'across' the text stretches, the more likely a 
vertical expansion will also result.


Some folk dislike the spaces, and claim the opposite effect. 
Strangely-enough, when I linked their coding views to their clothing 
choices (gaps, over-laps, ...) then it became a matter of 'personal 
freedom' and a cant that others/employers should not confuse outward 
appearance with (inner) competence.


Then again, notice my (optional) parentheses here, and how there's no 
spaces 'inside' them - which do appear in my Python. Inconsistency is a 
human condition. We are not machines.



A tool is something which will make one's life easier, but harmony 
within a team is an intangible value and much under-rated - until 
experienced!



On 22/10/19 3:49 AM, Paul Moore wrote:

IMO, if you care enough to not like black's formatting choices, you
probably shouldn't use it. The point of black is to *not* care about
formatting, but leave the decisions to the tool. If you're not doing
that, then it's probably the wrong tool for you. There may be other
code formatters that are more configurable - I honestly don't know -
and which therefore may be more useful to you once configured to your
liking. But black is (as I understand it) about *not* worrying about
formatting choices, so you can get on to other more important
decisions.

Personally, I don't really like some of black's choices. So I only use
it on projects where I'm tired of worrying about formatting and
readability and just want something that's OK. Over time, I'm finding
that's true of more and more of my projects :-)

Regarding your question on spaces, I like some of your spacing
choices, but not all of them. So we could have a debate over precisely
which spaces I agree with and which I don't. But I'm not really
interested in doing that. Black would probably help if we had to work
together, but unless we do, you can do what you want and I won't mind
:-)

Paul

On Mon, 21 Oct 2019 at 15:22,  wrote:


What do people think about black?

I'm asking because one of my personal preferences is to use spaces for clarity:

1.  right = mystr[ start : ]

  black version right=mystr[start:]

2.  mtime = time.asctime( time.localtime( info.st_mtime ) )

  black version mtime = time.asctime(time.localtime(info.st_mtime))

Is there a reason why PEP8 doesn't like these spaces?
--
https://mail.python.org/mailman/listinfo/python-list


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: TypeError: unhashable type: 'list'

[DL Neil via Python-list]

On 23/10/19 8:51 PM, joseph pareti wrote:

I am experimnenting with this (reproducer) code:
pattern_eur= ['Total amount']
mylines = []# Declare an empty list.
with open ('tmp0.txt', 'rt') as myfile: # Open tmp.txt for reading text.
 for myline in myfile:   # For each line in the file,
 mylines.append(myline.rstrip('\n')) # strip newline and add to list.
for element in mylines: # For each element in the list,
match_C = re.search(pattern_eur, element)
if match_C:
  element = element + 2
  print(element)
--
the input file being:
$ cat tmp0.txt
line 0
line 1
Total amount

5.00
linex

...

My intent is to locate the line containing "Total amount", skip the next
line, then print the eur value. The program terminates as follows:

...


Thanks for any insigths --



The first observation is that the two for loops are essentially 
identical, so why not condense?


However, what is described may be calling for a solution called "a 
finite state machine":


state 1: ignore unwanted data, until "Total amount" is found

state 2: skip blank line

state 3: grab the Euro value, and return to state 1


Being a simple-boy, I would avoid any reg-ex, because:

myline[ :11 ] == "Total amount"

is easier (and faster). Similarly, there is no need for rstrip-ping 
except at "state 3" (unless there are particular rules for the 
formatting of the total).



Another thought is that the problem is being visualised as a series of 
lines and this may complicate things. If instead, a "buffer" or indeed 
the entire file, could be read at a time (which is current code, per 
first comment above), the string.find() method could be employed 
(replacing "state 1"), and then (implicit assumption about spacing here) 
"state 2" becomes a matter of moving a few characters 'along', before 
grabbing the total; rinse and repeat...



Web-ref:
https://en.wikipedia.org/wiki/Finite-state_machine
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Black

[DL Neil via Python-list]

On 22/10/19 3:18 AM, lizhollinshe...@gmail.com wrote:

What do people think about black?

I'm asking because one of my personal preferences is to use spaces for clarity:

1.  right = mystr[ start : ]

  black version right=mystr[start:]

2.  mtime = time.asctime( time.localtime( info.st_mtime ) )

  black version mtime = time.asctime(time.localtime(info.st_mtime))

Is there a reason why PEP8 doesn't like these spaces?



FYI:
Python Bytes pod-cast "Episode #153: Auto format my Python please!"
https://pythonbytes.fm/episodes/show/153/auto-format-my-python-please
(topic nr6)
NB I have not listened to it.
[via PlanetPython]

Apparently informed by blog article: "written on 06/02/2018
Auto formatters for Python 👨‍💻🤖"
https://www.kevinpeters.net/auto-formatters-for-python

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Fursday Flippancy: American Py

[DL Neil via Python-list]

[via PlanetPython]

The "American Py" song.
Lyrics which amused me, at
https://www.reddit.com/r/Python/comments/dfm2zv/american_py/

'Multi-taskers' may like to read and listen-along to:
https://www.youtube.com/watch?v=uAsV5-Hv-7U


For the benefit of us silver-surfers reliving our youth
(or for those condemned to repeat history):
https://en.wikipedia.org/wiki/American_Pie_%28song%29

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Congratulations to @Chris

[DL Neil via Python-list]

Chris Angelico: [PSF's] 2019 Q2 Community Service Award Winner
http://pyfound.blogspot.com/2019/10/chris-angelico-2019-q2-community.html

...and for the many assistances and pearls of wisdom he has contributed 
'here'!

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Instantiating sub-class from super

[DL Neil via Python-list]

On 25/10/19 4:29 AM, Frank Millman wrote:

On 2019-10-19 12:37 AM, DL Neil via Python-list wrote:

On 16/10/19 6:33 PM, Frank Millman wrote:

On 2019-10-14 10:55 PM, DL Neil via Python-list wrote:
Is there a technique or pattern for taking a (partially-) populated 
instance of a class, and re-creating it as an instance of one of its 
sub-classes?


Here is a link to an article entitled 'Understanding Hidden 
Subtypes'. It dates back to 2004, but I think it is still relevant. 
It addresses precisely the issues that you raise, but from a 
data-modelling perspective, not a programming one.


http://tdan.com/understanding-hidden-subtypes/5193

I found it invaluable, and applied the concepts in my own 
business/accounting application. Having created the ability to make 
subtypes visible and explicit, I found all kinds of unexpected uses 
for them.


The article seems to be missing a couple of images (Figure 1 and 
Figure 2) showing the data relationships. I downloaded the original 
article onto my computer years ago, and my local copy does have the 
images, so if you would like to see them let me know and I will 
upload my version somewhere to make it accessible.


Superb!

Yes please Frank - I've also approached it from the data/DB side, and 
thus presumably why I was puzzling over how one implements in Python.


(alternatively, email a PDF/similar directly)


Hi

I have just got back from a few days break and have only seen your 
message now.


Did you see the message I posted on the 17th with a download link? If 
not, would you like me to post it again?


I did spot the later post - but only after I'd written the above.

I've been amused by the similarities between their case and ours, and 
enjoyed learning a lesson (or two?).


Thanks and apologies.
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Installing Python 3.8 on PC with Python 3.7. How to do with version conflicts?

[DL Neil via Python-list]

Dottore,


On 28/10/19 7:37 AM, Dott. Ugo Donini wrote:

I cannot use Python 3.8 on my PC with installed Python 3.7.
Is it possible to update the existing Python 3.7 without reinstalling Python 
3.8.
Conflicts problems.
Thankyou
Ugo Donini
Inviato da Posta per Windows 10



If the question is about updating Python 3.7 'in place' (and not 
upsetting any applications or systems programs which use Python), then 
"yes, this is possible": you may be able to update r3.7.0 to r3.7.4, for 
example (only the last digit changes). I see few compelling reasons to 
do-so, but YMMV (you have your own reasons).


If the question is a desire to run both Python 3.7 AND Python 3.8 on the 
same machine, then the PSL's (Python Standard Library) answer is "venv". 
Which you will find described in the docs (documentation) at "12. 
Virtual Environments and Packages".


For my own reasons, instead of Python virtual environments, I prefer to 
separate different clients/projects by keeping them in their own 
VirtualBox VM. Some dislike this particular solution as it lacks "open 
source" principles. There are other "container" technologies worth 
review. However, I suggest that this type of solution is the preserve of 
the more advanced programmer/computer user.



WebRefs:
https://docs.python.org/3/tutorial/venv.html
https://www.virtualbox.org/
--
Ciao =dn
--
https://mail.python.org/mailman/listinfo/python-list

Friday finking: TDD and EAFP

[DL Neil via Python-list]

Is the practice of TDD fundamentally, if not philosophically, somewhat 
contrary to Python's EAFP approach?



TDD = Test-Driven Development
EAFP = it's easier to ask forgiveness than permission
* WebRefs as footnote


The practice of TDD* is that one writes test routines to prove a unit of 
code, eg method or function; BEFORE actually writing said function.


The rationale is that TDD encourages proper identification and 
consideration of the routine's specification, and attempts to ensure 
that exceptions and "edge-cases" are not quietly forgotten.

(in a broad-brush, nut-shell)

However, I quite possibly like yourself, come from a time-before - 
before TDD, and before Python. So, have had to not only learn these 
things, but sometimes, un-learn points and habits (if not vices). 
Accordingly, I came (quite unknowing of the term, or that there might be 
an alternative) from the world of LBYL* (look before you leap).


In other words, before you do anything with some data, check that it is 
what you think it is. Whereas in Python we "try" by assuming everything 
is compos-mentis* and handle the "except" when things are not how we'd like.


That adaptation was not too difficult. After all, aren't programmers an 
optimistic bunch - I mean, I *never* introduce bugs into *my* code! Do you?


Which brings us to TDD. Here we assume the likelihood of bugs, as-if 
(cue: manic giggling); and code a bunch of tests first - in an attempt 
to prove that the code is up-to-spec.


In encouraging my mind to think about testing the code, I find myself 
searching for edge-cases, and attempting to anticipate the unusual. 
Accordingly to the gospel of TDD: so far, so good.


The 'problem' is, that it puts my mind onto LBYL-rails before the 
Python-coding train (of thought) has even left the station. It then 
seems natural to start putting a bunch of if-then-else's up-front and 
before the 'main line' of code.


Does TDD bend your mind in this (apparently) non-Pythonic fashion?
Have you developed an answer?
(other than: "@dn is 'nuts'*", which is not worthy of debate)


WebRefs:
https://duckduckgo.com/?q=TDD&ia=web
https://devblogs.microsoft.com/python/idiomatic-python-eafp-versus-lbyl/
https://docs.python.org/3/glossary.html#term-eafp
but: https://mail.python.org/pipermail/python-dev/2014-March/133118.html
https://docs.python.org/3/glossary.html#term-lbyl
Latin/legal term "compos mentis" 
https://www.thefreedictionary.com/compos+mentis

English slang term "nuts": https://www.thefreedictionary.com/nuts
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How can i stop this Infinite While Loop - Python

[DL Neil via Python-list]

TLDR; declare if homework; doing someone's homework doesn't really help; 
Python is not ALGOL/Pascal/C/C++ by any other name; Python assignments 
should promote learning semantics as well as syntax; sometimes 
re-stating the problem leads to an alternate/better solution.



On 31/10/19 12:55 AM, ferzan saglam wrote:

I have tried many ways to stop the infinite loop but my program doesn't seem to 
stop after 10 items! It keeps going...!)

---
total = 0
while True:

   print('Cost of item')
   item = input()

   if item != -1:
 total += int(item)
   else:
 break

print(total)
---



A critique of this thread, plus a suggested approach to a solution:-


1 the OP
This is a contrived coding problem. Is it part of a course? To establish 
an honest relationship with your list-colleagues, should you have 
declared it as 'homework'?


Remembering that no-one 'here' is paid to be helpful, did you politely 
respond with the clarification requested?

(see also later threads)

There is an interesting psychology that you will notice when working 
with other people who are happy to be helpful (per this list-community); 
that if one stops to think about *how* to pose a question, sometimes the 
answer 'appears' - a burst of inspiration in your mind.


That can be difficult to remember if you've been struggling with a 
problem for hours with the resultant level of frustration.


At least the title of the post: "How can i stop this Infinite While 
Loop" is descriptive. Compared with some first-time poster's titles, 
this is clear and competent. Thanks!


Thinking about it though, is there a hint of the nature of the problem, 
or even then an answer, in there? ie pausing to think about it, is the 
problem actually the "stop" (as emphasised) or might it be the "while"? 
(more below)



2 respondents
This was an 'easy answer' question, so "well done" and "thank you" for 
jumping-in with the intention of contributing something to the community 
- quite possibly a first post for some. I encourage you to participate 
and do NOT wish to criticise you for having the best of motivations.


However, it is always a good idea to consider not only "the question", 
as asked, but to 'first ask why?' (which happens to (almost) be the 
title of a thoroughly-recommendable book for managers, 
potential-managers, and certainly coaches/mentors to read). You can see 
this in one respondent's <>>


If (and this is *my* assumption!), the OP is a student attempting to 
learn Python, cf an experienced pythonista; giving "the answer" is NOT 
as helpful as it might seem. That's why the "pseudo code" solution was 
'better' than a Python equivalent.


What should the OP learn? Given 'the answer', would the OP be able to 
solve a similar problem when next encountered?


If the OP's learning-objective is 'Python', then the pseudo-code helped 
clarify the necessary thinking, without detracting from the OP's 
learning-opportunity. (cf an algorithms course)


In fact, and quite coincidentally, the best learning-opportunity here, 
has probably been the "off by one" experience - which the OP did indeed 
learn for himself. (Well done!)



3 the question itself/the questioner
The way the OP has attempted to solve this contrived-problem indicates a 
fundamental lack of understanding of the way Python works and/or the way 
pythonista think. However, the way the question has been posed indicates 
much the same! It smacks of an old, even ancient, ComSc problem; 
previously applied to other languages (something I might have written 
when learning/teaching FORTRAN more than four decades ago, for example). 
How does this coding-problem contribute to the *Python* learning 
process? (as well as contributing towards the learning ComSc principles, 
eg managing/terminating loops)



Thus:


The output I am aiming for is for the code to stop asking 'Cost of item' after 
10 inputs or until -1 is typed, but the code repeats itself on an infinite 
loop. I cannot seem to stop the infinite loop.


expresses the problem, but in the manner of other languages. The focus 
has been put on the "-1" fact - how to "stop". (remember the OP's title!)


Question: since we no longer present data as decks of punched-cards, 
what is today's more usual way to indicate EOF/EOD? ("end of file", "end 
of data") Is the minus-one "flag" a very good approach, or even 
particularly representative of work-place 'realities'?



Towards a solution:

OK, let's not argue the point - it's certainly beyond the OP's control, 
so we must consider it a "specification"...


The advice for solving ComSc (and other) problems is always to 'break 
down the larger problem into smaller, sub-problems and solve them' 
(which may, in-turn, involve breaking the sub-problems into smaller...)


Here 'the problem' may decompose into:
1 accept input value
2 accumulate (total) input data-values
3 

Re: Friday finking: TDD and EAFP

[DL Neil via Python-list]

On 3/11/19 6:30 AM, Bev In TX wrote:
On Nov 1, 2019, at 12:40 AM, DL Neil via Python-list 
mailto:python-list@python.org>> wrote:


Is the practice of TDD fundamentally, if not philosophically, somewhat 
contrary to Python's EAFP approach?


I’m not an expert on either TDD or Python, but as no one else has 
responded, I’ll put in my 2 cents worth...
You are mixing paradigms.  TDD is about testing code, regardless of the 
style in which it was written.


Agreed: (in theory) TDD is independent of language or style. However, 
I'm wondering if (in practice) it creates a mode of thinking that pushes 
one into an EAFP way of thinking?



Yes, exceptions can handle some edge cases; however, that does NOT mean 
that ...


  * all code handles all edge cases.
  * all bugs are only in edge cases.
  * all exception handling code is perfectly written.
  * Etc


The preceding description was deliberately short and not intended as a 
definition of TDD, nor bug-hunting. Have I missed your point here? 
(apologies)


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday finking: TDD and EAFP

[DL Neil via Python-list]

On 2/11/19 4:32 PM, boB Stepp wrote:

On Fri, Nov 1, 2019 at 12:42 AM DL Neil via Python-list
 wrote:


Is the practice of TDD fundamentally, if not philosophically, somewhat
contrary to Python's EAFP approach?

[...]

In encouraging my mind to think about testing the code, I find myself
searching for edge-cases, and attempting to anticipate the unusual.
Accordingly to the gospel of TDD: so far, so good.

The 'problem' is, that it puts my mind onto LBYL-rails before the
Python-coding train (of thought) has even left the station. It then
seems natural to start putting a bunch of if-then-else's up-front and
before the 'main line' of code.

Does TDD bend your mind in this (apparently) non-Pythonic fashion?
Have you developed an answer?
(other than: "@dn is 'nuts'*", which is not worthy of debate)


As the pros have yet to chime in, I will attempt to add my amateurish


Thanks!

Because I have several rôles and spend only a proportion of my time 
actually coding, I fear that I should claim a less-than professional 
status. No matter.


It's an interesting comment though: I've been wondering if many 'Python 
pros' actually use TDD - few of the people I work alongside (in 
programmer-mode) do. Thus there are TDD 'evangelists', loud detractors, 
and most 'in the middle' or simply not wanting to engage in such a debate.


Perhaps 'Python pros' don't use TDD. Which in-and-of-itself, somewhat 
answers the question!




thoughts.  In my on again/off again efforts to self-study Python,
sound software construction practices, etc., I have been endeavoring
to adopt a TDD approach.  I find that trying to do this forces upon me
more clarity of thought which (hopefully) results in more expressive,
simpler and understandable code.  As I often find my initial efforts
at writing tests murky and hard to understand, I (eventually) realize
that the associated code I'm trying to test, which seemed so clear
when I first wrote it, is hard to test.  So I refactor/simplify the
code, it becomes easier to write sensible test code, etc.  It may just
be me and my lack of experience, but I find this circular process
ultimately leads to more Pythonic results -- easier to understand code
that (I hope) others can easily read and understand.

Though I am sure I am stating the obvious, instead of just thinking
about edge cases, I like to have at least one test that proves that
what I normally expect to get is, in fact, gotten.  I often
embarrassingly find myself caught in some sort of simple logic error
that this type of test will catch, especially later if I am
refactoring my code in the direction I am trying to coerce it to go
for the end product.

I am probably far from thinking/coding in a proper Pythonic fashion,
but the TDD back-and-forth process seems to get me closer to being
Pythonic.


Yes, normally the first test(s) (eg for some method/function) would be 
the desired behavior(s) - testing that we will enjoy the expected 
benefits of the actual code/that the code does what it should. 
Thereafter it become a case of ensuring there's nothing 'else' or 
undesired... ie there's no point in worrying about (say) a "divide by 
zero" condition, if the formula doesn't already prove with valid data!


Unlike yourself - and probably the circular approach described by most 
TDD advocates, my approach has been to write the method's (probably) 
signature and docstring + pass first. Then to transfer those objectives 
to Pytest code, where I write test-after-test - because I'm in the mode 
of thinking about 'what-ifs'.


Once I'm reasonably satisfied that I've covered the specs, then it 
becomes time to write the code - during which I do follow a circular 
fashion - satisfy the first test, then satisfy the second... always 
ensuring that subsequent refactoring doesn't abrogate an earlier 'pass'.



Hence the observation that an (attempt at an) exhaustive examination of 
the spec is rather LBYL; whereas the circular code-and-test might be 
more Pythonic if it were more EAFP, but that starting with a somewhat 
LBYL approach might tend to keep one thinking that way...


(of course it is never quite a clean-cut the idealised two-stage 
description (above). There are always considerations which occur to me 
'later', and changes/improvements to the spec which arise as the team 
'burns down' the sprint...)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looking for python pentest scripts

[DL Neil via Python-list]

On 11/11/19 12:36 AM, nixuser wrote:

can someone tell about good resource for python related pentesting
scripts?
any extensive list?



What is the purpose of such scripts/list?

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Looking for python pentest scripts

[DL Neil via Python-list]

On 11/11/19 4:39 PM, Terry Reedy wrote:

On 11/10/2019 7:32 PM, Bob Gailer wrote:

On Nov 10, 2019 6:40 AM, "nixuser"  wrote:

can someone tell about good resource for python related pentesting
scripts?
any extensive list?

 > Try Googling python pentesting. That will give you some relevant links.
(Google's Youtube has multiple sometimes interesting videos where people 
do penetration tests with bullets and arrows and such against various 
targets.  It turns out the the 'funny-looking' extensions near the top 
of some medieval plate armor chest pieces serve to block arrow heads 
deflected up off the chest toward the chin.)



TLDR? Last paragraph is most pertinent.


I read this, whilst still chortling at the opus @Chris had pen-ned about 
testing.


Then one of the guys from my old unit rang. He was grumbling (even 
though Americans are typically less familiar with the British 
expression: grumpy, old men - it suits us). He wanted to know if we have 
been "infected" over here (we have), his complaint was about the rising 
fashion for "Singles Day", and "doesn't 11 and 11 mean either 22 or 4? 
So, what's 'single' about that?". Guess he has a point, how come 'they' 
didn't at least 'downgrade' to choosing 1/1? Some (other) significance 
to the numbers perhaps?


Evidently Remembrance Day/Veterans' Day can have different meanings, as 
per "pen-testing".


Similarly, an irony: the OP hasn't fronted to say whether (s)he has 
'white hat' or 'black hat' ambitions; just as the contemplation of war 
and peace brings one to consider how the motives behind the possibility 
of single-selfishness today, contrast markedly from those of accepting 
the suffering and sacrifice which helped to make this world what it is.


To distract from the above conversational tone, I mentioned the 'armor' 
deflection-pieces (there's probably a name for these, but we didn't use 
"armor" (I'm not THAT old) - nor did we have kevlar vests!). The laconic 
reply was that "No, but when you're the target of attack, we'd 
burrow-down attempting to hide behind the smallest pebble". Almost the 
same thing then?



However, we're here to discuss Python:
A 'security' colleague commented that there is a Linux 
distro/distribution, "Kali" (another name with opportunity for 
ambiguity!). Apparently, "Kali is designed for pen-testing - and there 
are likely many associated Python scripts, because Python is a language 
so well suited to fast-developing situations!". YMMV...



WebRefs:
https://en.wikipedia.org/wiki/Remembrance_Day
https://en.wikipedia.org/wiki/Kali
https://www.kali.org/
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Hi there! We are here to answer any questions you have about Udacit...

[DL Neil via Python-list]

On 12/11/19 7:14 AM, joseph pareti wrote:

i have done the first 6 lessons of python ---
https://classroom.udacity.com/courses/ud1110/lessons/bbacebc6-406a-4dc5-83f6-ef7ba3371da6/concepts/50247542-7933-4afe-9130-ff1dff429b03

what do you recommend next? My goal is ML/AI



As with any professional field, there is no end to the learning...

Probably some consolidation to provide a solid set of Python skills, 
would be a good idea; and then to dig-into ML - if you have the patience 
and long-term view.


If you'd like to switch to books:
There are plenty to choose from. My impression of the current crop of 
Python/ML/AI books is that they either start at a reasonably 'high' 
level or their 'intro to Python' chapters are rather perfunctory. Have 
you an assessment?
Another impression, is that in the rush-to-publish/to be 'first', there 
is a very confusing coverage. Perhaps better to read non-specific books 
about ML/AI as preparation, and then focus on specifics (and thus 
applicable Python texts)?
Meantime, build-up your Python skills - I've previously mentioned 'here' 
the The Python Craftsman series (Apprentice, Journeyman, Master), but 
there are others.


If you're comfortable with MOOCs:
I have found Udacity courses tend to be a little 'thin' (YMMV!).
Coursera has a range of offerings, and you might review the multiple 
series from U.Michigan (have assessed some and are IMHO competent but 
not earth-shattering).
U.Mich also offer a number of ML courses/quals, which may be of-interest 
(and hence the specific mention).
Similarly, edX where I noted a number of IBM ML offerings both with and 
without Python and likely majoring on "Watson".
(apologies, I'm biased against the multitude of MSFT's offerings, also 
present)



WebRefs:
https://leanpub.com/b/python-craftsman
(per example) https://www.coursera.org/specializations/data-science-python
https://www.edx.org


Disclaimer: have no commercial connection to any of the above-mentioned, 
but do use the edX platform to offer (non-Python) training.

--
Tschüss =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: apologies for my latest email; it was not intended for this mailing list

[DL Neil via Python-list]

Because of something we said?

(to upset you=joke!)
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Hi there! We are here to answer any questions you have about Udacit...

[DL Neil via Python-list]

On 12/11/19 9:48 PM, joseph pareti wrote:
great, thank you so much for the advice. In fact, I sent this mail to 
the python mailing list by mistake, but now I am glad I did ...


There's plenty of over-lap between lists - PyTutor is another.

Meantime I've received email from IBM about their ML/AI video series, 
which may also interest you. Today/tomorrow they are offering a 
web-cast: An Introduction to Visual Recognition.

https://developer.ibm.com/events/an-introduction-to-visual-recognition-11-13-2019/

This invitation came from one of their email subscriptions to which you 
could also subscribe. Undoubtedly their bias is towards "Watson". Some 
activities enable free access to a Watson cloud...


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday finking: TDD and EAFP

[DL Neil via Python-list]

Apologies for lateness - stuff happened...


On 4/11/19 9:44 AM, Peter J. Holzer wrote:

On 2019-11-04 07:41:32 +1300, DL Neil via Python-list wrote:

On 3/11/19 6:30 AM, Bev In TX wrote:

On Nov 1, 2019, at 12:40 AM, DL Neil via Python-list
mailto:python-list@python.org>> wrote:

Is the practice of TDD fundamentally, if not philosophically,
somewhat contrary to Python's EAFP approach?



Agreed: (in theory) TDD is independent of language or style. However, I'm
wondering if (in practice) it creates a mode of thinking that pushes one
into an EAFP way of thinking?


This is exactly the opposite of what you proposed in your first mail,
and I think it is closer to the truth:


It is a while ago, and I cannot honesty remember if I was attempting to 
provoke discussion/debate/illustration by switching things around, or if 
I simply confused the abbreviations.




TDD does in my opinion encourage EAFP thinking.

The TDD is usually:

 1 Write a test
 2 Write the minimal amount of code that makes the test pass
 3 If you think you have covered the whole spec, stop, else repeat
   from 1

This is often (e.g. in [1]) exaggerated for pedagogic and humoristic
reasons. For example, your first test for a sqrt function might be
 assert(sqrt(4) == 2)
and then of course the minimal implementation is
 def sqrt(x):
 return 2


I have seen this sort of thing in spreadsheet training - someone 
pulling-out a calculator, summing a column of numbers, and typing 'the 
answer' in the "Total" cell (instead of using the Sigma button or @SUM() ).


However, I've never seen anyone attempt to pass-off this sort of code 
outside of desperate (and likely, far too late) floundering during a 101 
assignment - FAILED!


Who would begin to believe that such code implements sqrt, or that it 
meets with the function's objectives as laid-out in the spec AND the 
docstring? So, anyone can prove anything - if they leave reality/reason 
far-enough behind.


Unfortunately the phrase "test pass" (as above) can be misused?abused in 
this way; but nowhere in your or my descriptions of TDD did *we* feel it 
necessary to point-out that the/any/every test should be true to the 
spec. It's unnecessary.


Great joke, but its proponents are delaying a proper consideration of 
TDD. Perhaps they are hiding behind something?




Which just means that we don't have enough test cases yet. But the point
is that a test suite can only check a finite (and usually rather small)
number of cases, while most interesting programs accept a very large (if
not really infinite) number of inputs, so the test suite will always be
incomplete. At some point you will have to decide thet the test suite is
good enough and ship the code - and hope that the customer will forgive
you if you have (inevitably) forgotten an important case.


Which highlights a difference between the description (above) and the 
approach I described: I tend to write 'all' the tests first - which, now 
that I type it, really does sound LBYL!


Why? Because testing is what we might call a diagnostic mode of thinking 
- what is happening here and why? Well, nothing is actually happening in 
terms of the application's code; but I'm trying to think-ahead and 
imagine the cases where things might not go according to plan.


Later, I shift to (what?) code-authoring/creative mode (yes, someone's 
going to point-out the apparent dichotomy in these words - please 
go-ahead) and 'bend the computer to my will'. It is the tests which 
(hopefully) prove the success or otherwise of my output. I'm not coding 
to the test(s) - sometimes I deliberately take a meal-break or 
overnight-break between writing tests and writing code.


Why not follow 'the letter of the law'? Because it seems to me that they 
require different ways of looking (at the same problem). My fear of 
doing them one-at-a-time is that I'll conclude (too early) exactly as 
you say - that's enough testing, 'it works'!



Returning to my possible re-interpretation/misuse of the TDD paradigm: 
I'm only officially 'testing' one test at a time, but if 'this iteration 
of the code' passes two or more of the next tests in the series, well... 
("oh what a good boy am I"!) In practice, I'll review this iteration's 
test and its motivation(s) to ensure that the code hasn't passed 'by 
accident' (or should that be, by "bi-product").


Now, reviewing the: write one (useful/meaningful) test, then write the 
code until it passes (this and all previous tests), then re-factor 
(tests and application code), rinse-and-repeat. The question is: might 
the 'diagnostic' mode of thought 'infect' the 'creative' and thereby 
encourage writing more "defensive code" than is EAFP/pythonic?




There is very li

Re: Friday finking: TDD and EAFP

[DL Neil via Python-list]

On 6/11/19 8:01 AM, Barry Scott wrote:

On 1 Nov 2019, at 05:40, DL Neil via Python-list  wrote:

Is the practice of TDD fundamentally, if not philosophically, somewhat contrary 
to Python's EAFP approach?
The practice of TDD* is that one writes test routines to prove a unit of code, 
eg method or function; BEFORE actually writing said function.
The rationale is that TDD encourages proper identification and consideration of the 
routine's specification, and attempts to ensure that exceptions and 
"edge-cases" are not quietly forgotten.
(in a broad-brush, nut-shell)


The practice of creating software is a social activity where those involved
have foibles. Managing all the social interactions and foibles is what
the methodologies are trying to help with. Any methodology that is easier
to follow then avoid will tend to be taken up and provide a benefit.

We all know that tests are a must have, but often those tests that we all
know are a must have do not get written. It can often seem like a chore,
after all the code works right?

By starting with the tests the social engineering means that you make having
the tests the easy part of the process.

Methodologies like Agile address other foibles. Given a far off dead line
the tendency is to delay getting the bulk of the work done until at the last
moment.

So is TDD contrary to EAFP? Not as such its two sorts of social engineering.
TDD helps get the tests, EAPF give permission to take risks, necessary to
innovate.


+1
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Writing a CPython extension - calling another sibbling method ?

[DL Neil via Python-list]

On 20/11/19 9:20 AM, Luciano Ramalho wrote:

I apologize to all but the intended recipient for this. I’d have given him
feedback in private if I knew his email.

I will take leave from the list now. Keep up the good work, friendly
responders.



Please reconsider. Should your relationship with the 'good folks' be 
governed by someone/anyone else?



There is no need to tell you that are many innocent reasons why we 
misunderstand each other's motives or intent, particularly in a 
multi-cultural context; just as there are folk who attempt to 'take 
advantage'. Grossly generalised examples
- each academic year a new crop of PyStudents seems to crop-up, thinking 
we'll *do* their 'homework' for them...

- some parts of British humor is impenetrable to those born south of Dover
- Dutch/German positivity can be interpreted as brusque or even 
arrogant, in some parts of the world
- Kiwi/Aussie humor as often involves insult (which is not meant to be 
taken seriously) as it does self-deprecation

- Self-deprecation is not 'the American way'...
- Monty Python humor is its own (lack of) explanation!

That said, I am asking (elsewhere, but please feel free...'here') if the 
standards and conventions of the Python Foundation apply to this/these 
list(s)? Such requires a balance between an individual's right to 
privacy, and the open-ness with which we conduct Python-business, ie 
without insult, derision, victimisation, etc, etc...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python Resources related with web security

[DL Neil via Python-list]

Curiosity: why have recent similar enquiries also come from 
non-resolving domain names?



Recently we've seen security-related enquiries (on more than one Python 
Discussion List) which don't explicitly claim to come from 'white hat 
hackers' but which do have the potential to have less-than productive 
aims or interests.


Are such email addresses 'open' and honest?
Do they create extra and unnecessary scut-work for ListAdmins?
Does such meet PSF 'standards' for honest and respectful interchange?

WebRef:
https://www.python.org/psf/conduct/



On 24/11/19 8:18 AM, Pycode wrote:

Hello,

can anyone post links for python resources that contain tools and scripts
related with security and pentesting?
not looking for the obvious such as OWASP,etc

can anyone post a list of interesting links? you can also include blogs
and forums..

Thanks



--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.8 with Anaconda

[DL Neil via Python-list]

On 25/11/19 6:26 AM, Ugo Donini wrote:

It is lossible to update Anaconda to Pytho. 3.8?



First 'hit' on DuckDuckGo.com: 
https://www.anaconda.com/keeping-anaconda-date/


If it is not yet available, they are the ones to ask if/when a 3.8 
collection will be assembled...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python Resources related with web security

[DL Neil via Python-list]

On 26/11/19 11:48 AM, Tim Chase wrote:

On 2019-11-25 21:25, Pycode wrote:

On Sun, 24 Nov 2019 10:41:29 +1300, DL Neil wrote:

Are such email addresses 'open' and honest?


you are not being helpful or answer the question..


What DL Neil seems to be getting at is that there's been an uptick in
questions

1) where we don't know who you (and several other recent posters) are:

   - The pyc.ode domain-name of your email address isn't a
 real/registered domain

   - there doesn't seem to be much evidence of you being part of the
 Python community with a history of other messages


+other consideration regarding the best use of mail-lists and respecting 
people's time.


For reference: an attempt to contact the OP directly, in a bid to refine 
his questions/assist the production of answers (perhaps), failed - due 
to the problem mentioned, ie one of his/her own making!




   Neither factor inspires much confidence.

2) you (and others) are asking to be spoonfed example code that could
cause problems on the internet.

...



So I suspect DL Neil was raising awareness to make sure that anybody
who *did* answer your questions might take the time to think about
the potential consequences of the actions.  So DL *is* being helpful,
but rather to the community, even if not necessarily to you in
particular


Thanks @tkc. Yes!

Given that the OP may have legitimate (and legal) reasons for the 
question, I was reluctant to express direct criticism (which, 
ironically-enough) is contrary to 'open' etc behavior.


Previous to this, had discussed such concerns with a ListAdmin, who is 
taking it further. I hope we'll (soon) see something further from the 
Python-/List-gods...



As a student of irony, I was amused at being told that I wasn't 
answering questions - by someone who didn't even acknowledge, and 
certainly didn't attempt to address, concerns raised. Colors += mast?


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Extract sentences in nested parentheses using Python

[DL Neil via Python-list]

On 3/12/19 6:00 AM, Peter Otten wrote:

A S wrote:
I think I've seen this question before ;)


In addition to 'other reasons' for @Peter's comment, it is a common 
ComSc worked-problem or assignment. (in which case, we'd appreciate 
being told that you/OP is asking for help with "homework")




I am trying to extract all strings in nested parentheses (along with the
parentheses itself) in my .txt file. Please see the sample .txt file that
I have used in this example here:
(https://drive.google.com/open?id=1UKc0ZgY9Fsz5O1rSeBCLqt5dwZkMaQgr).

I have tried and done up three different codes but none of them seems to
be able to extract all the nested parentheses. They can only extract a
portion of the nested parentheses. Any advice on what I've done wrong
could really help!


One approach is to research in the hope that there are already existing 
tools or techniques which may help/save you from 'reinventing the wheel' 
- when you think about it, a re-statement of open-source objectives.


How does the Python interpreter break-down Python (text) code into its 
constituent parts ("tokens") *including* parentheses? Such are made 
available in (perhaps) a lesser-known corner of the PSL (Python Standard 
Library). Might you be able to use one such tool?


The ComSc technique which sprang to mind involves "stacks" (a LIFO data 
structure) and "RPN" (Reverse Polish Notation). Whereas we like people 
to take their turn when it comes to limited resources, eg to form a 
"queue" to purchase/pay for goods at the local store, which is "FIFO" 
(first-in, first-out); a "stack"/LIFO (last-in, first-out) can be 
problematic to put into practical application. There are plenty of 
Python implementations or you can 'roll your own' with a list. Again, 
I'd likely employ a "deque" from the PSL's Collections library (although 
as a "stack" rather than as a "double-ended queue"), because the 
optimisation comes "free". (to my laziness, but after some kind soul 
sweated-bullets to make it fast (in both senses) for 'the rest of us'!)




It's probably easier to understand and implement when you process the
complete text at once. Then arbitrary splits don't get in the way of your
quest for ( and ). You just have to remember the position of the first
opening ( and number of opening parens that have to be closed before you
take the complete expression:


+1
but as a 'silver surfer', I don't like to be asked to "remember" anything!



level:  000100
 we need^



Consider:
original_text (the contents of the .txt file - add buffering if volumes 
are huge)

current_text (the characters we have processed/"recognised" so-far)
stack (what an original name for such a data-structure! Which will 
contain each of the partial parenthetical expressions found - but yet to 
be proven/made complete)


set current_text to NULSTRING
for each current_character in original_text:
if current_character is LEFT_PARENTHESIS:
push current_text to stack
set current_text to LEFT_PARENTHESIS
concatenate current_character with current_text
if current_character is RIGHT_PARENTHESIS:
# current_text is a parenthetical expression
# do with it what you will
pop the stack
set current_text to the ex-stack string \
concat current_text's p-expn

Once working: cover 'special cases' (after above loop), eg original_text 
which doesn't begin and/or end with parentheses; and error cases, eg 
pop-ping a NULSTRING, or thinking things are finished but the stack is 
not yet empty - likely events from unbalanced parentheses!


original text = "abc(def(gh))ij"

event 1: in-turn, concatenate characters "abc" as current_text
event 2: locate (first) left-parenthesis, push current_text to stack(&)
event 3: concatenate "(def"
event 4: push, likewise
event 5: concatenate "(gh"
event 6: locate (first) right-parenthesis (matches to left-parenthesis 
begining the current_string!)

result?: ?print current_text?
event 7: pop stack and redefine current_text as "(def(gh)"
event 8: repeat, per event 6
event 9: current_text will now become "(def(gh))"
event 10: (special case) run-out of input at "(def(gh)ij"
event 11: (special case) pop (!any) stack and report "abc(def(gh)"


NB not sure of need for a "level" number; but if required, you can infer 
that at any time, from the depth/len() of the stack!


NBB being a simple-boy, my preference is for a 'single layer' of code, 
cf @Peter's generator. Regardless the processes are "tokenisation" and 
"recognition".


At the back of my mind, was the notion that you may (eventually) be 
required to work with more than parentheses, eg pair-wise 
square-brackets and/or quotation-marks. In which case, you will need to 
also 'push' the token and check for token-pairs when 'pop-ping', as well 
as (perhaps) recognising lists of tokens to tokenise instead of the two 
parenthesis characters alone. In which case, I

Re: Unicode filenames

[DL Neil via Python-list]

On 7/12/19 7:17 AM, Bob van der Poel wrote:

I have some files which came off the net with, I'm assuming, unicode
characters in the names. I have a very short program which takes the
filename and puts into an emacs buffer, and then lets me add information to
that new file (it's a poor man's DB).

Next, I can look up text in the file and open the saved filename.
Everything works great until I hit those darn unicode filenames.

Just to confuse me even more, the error seems to be coming from a bit of
tkinter code:
  if sresults.has_key(textAtCursor):
 bookname = os.path.expanduser(sresults[textAtCursor].strip())

which generates

   UnicodeWarning: Unicode equal comparison failed to convert both arguments
to Unicode - interpreting them as being unequal  if
sresults.has_key(textAtCursor):

I really don't understand the business about "both arguments". Not sure how
to proceed here. Hoping for a guideline!



(I'm guessing that) the "both arguments" relates to expanduser() because 
this is the first time that the fileNM has been identified to Python as 
anything more than a string of characters.


[a fileNM will be a string of characters, but a string of characters is 
not necessarily a (legal) fileNM!]


Further suggesting, that if you are using Python3 (cf 2), your analysis 
may be the wrong-way-around. Python3 treats strings as Unicode. However, 
there is, and certainly in the past, was, no requirement for OpSys and 
IOCS to encode in Unicode.


The problem (for me) came from MSFT's (for example) many variations of 
ISO-8859-n and that there are no clues as to which of these was used in 
naming the file, and thus many possibly 'translations' into Unicode.


You can start to address the issue by using Python's bytes (instead of 
strings), however that cold reality still intrudes.


Do you know the provenance of these files, eg they are in French and 
from an MS-Win machine? If so, you may be able to use decode() and 
encode(), but...


Still looking for trouble? Knowing a fileNM was in Spanish/Portuguese I 
was able to take the fileNM's individual Unicode characters/surrogates 
and subtract an applicable constant, so that accented letters fell 
'back' into the correct Unicode range. (this is extremely risky, and 
could quite easily make matters worse/more confusing).


I warn you that pursuing this matter involves disappearing down into a 
very deep 'rabbit hole', but YMMV!


WebRefs:
https://docs.python.org/3/howto/unicode.html
https://www.dictionary.com/e/slang/rabbit-hole/
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Make warning an exception?

[DL Neil via Python-list]

On 7/12/19 9:58 AM, Israel Brewster wrote:

I was running some code and I saw this pop up in the console:

2019-12-06 11:53:54.087 Python[85524:39651849] WARNING: nextEventMatchingMask 
should only be called from the Main Thread! This will throw an exception in the 
future.

The only problem is, I have no idea what is generating that warning - I never 
call nextEventMatchingMask directly, so it must be getting called from one of 
the libraries I’m calling. Is there some way I can force python to throw an 
exception now, so my debugger can catch it and let me know where in my code the 
originating call is? I’ve tried stepping through the obvious options, with no 
luck so far.



We are able to "filter" errors, including turning warnings into 
full-bore errors. Of possible use: 
https://docs.python.org/3/library/warnings.html


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Error getting data from website

[DL Neil via Python-list]

On 7/12/19 12:53 PM, Sam Paython wrote:

This is the code I am writing:
import requests
from bs4 import BeautifulSoup
request = requests.get("https://www.amazon.ca/dp/B07RZFQ6HC";)
content = request.content
soup = BeautifulSoup(content, "html.parser")
element = soup.find("span",{"id":"priceblock_dealprice"})
print(element.text.strip())

and this is the error I am getting:
C:\Users\Sam\PycharmProjects\untitled2\venv\Scripts\python.exe 
C:/Users/Sam/PycharmProjects/untitled2/src/app.py
Traceback (most recent call last):
   File "C:/Users/Sam/PycharmProjects/untitled2/src/app.py", line 9, in 
 print(element.text.strip())
AttributeError: 'NoneType' object has no attribute 'text'

Could someone please help?



The err.msg/stack-trace is your friend! The comment about "NoneType" 
means 'there's nothing there' (roughly!) to print().


The question then becomes: "why?" or "why not?"...

With a short piece of code like this, and (I am assuming) trying-out a 
library for the first time, may I recommend that you use the Python 
REPL, because it allows you to 'see' what's going-on behind the 
scenes/underneath the hood - and ultimately, reveals the problem.


From a Python terminal (cmd is appropriate to your PC's OpSys):

[dn@JrBrown ~]$ python3
Python 3.7.4 (default, Jul  9 2019, 16:48:28)
[GCC 8.3.1 20190223 (Red Hat 8.3.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> from bs4 import BeautifulSoup
>>> request = requests.get("https://www.amazon.ca/dp/B07RZFQ6HC";)
>>> request# notice how I'm asking to 'see' what happened

>>> content = request.content
>>> content# there is no need to enclose in print()!
b'\n

Re: Error getting data from website

[DL Neil via Python-list]

On 7/12/19 1:51 PM, Chris Angelico wrote:

On Sat, Dec 7, 2019 at 11:46 AM Michael Torrie  wrote:


On 12/6/19 5:31 PM, DL Neil via Python-list wrote:

If you read the HTML data that the REPL has happily splattered all over
your terminal's screen (scroll back) (NB "soup" is easier to read than
is "content"!) you will observe that what you saw in your web-browser is
not what Amazon served in response to the Python "requests.get()"!


Sadly it's likely that Amazon's page is largely built from javascript.
So scraping static html is probably not going to get you where you want
to go.  There are heavier tools, such as Selenium that uses a real
browser to grab a page, and the result of that you can parse and search
perhaps.


Or look for an API instead.



Both +1
However, Selenium is possibly less-manageable for a 'beginner'.
(NB my poorly-based assumption of OP)

Amazon's HTML-response actually says this/these, but I left it open as a 
(learning) exercise for the OP. They likely prefer the API approach, 
because it can be measured...


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Unicode filenames

[DL Neil via Python-list]

On 8/12/19 5:50 AM, Bob van der Poel wrote:

On Sat, Dec 7, 2019 at 4:00 AM Barry Scott  wrote:

On 6 Dec 2019, at 18:17, Bob van der Poel  wrote:

I have some files which came off the net with, I'm assuming, unicode
characters in the names. I have a very short program which takes the
filename and puts into an emacs buffer, and then lets me add information

to

that new file (it's a poor man's DB).

Next, I can look up text in the file and open the saved filename.
Everything works great until I hit those darn unicode filenames.


...


Do you get the error with python 3?

I'm running this program on Linux (Ubuntu 19.10) and Python2.


...


I've taking the coward's way out and renamed the 1/2 dozen files. Seems
that it is when I grab a filename from the DB it is in unicode and the the
textAtCursor() and then I am trying to open that file using a fork to a
pdf-display program. This is all Q&D stuff so I'm going to file it under
"mysteries of life" and live with it :)



Fair enough, for such small number no other solution could be as 
efficient! My quick-and-dirty 'solution' would only work for (very few) 
'old data files' being recognised/name-updated using Python3.



Insert here: obligatory warning about the deprecation of Python2 at the 
end of this month/year...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Aw: Re: stuck on time

[DL Neil via Python-list]

On 8/12/19 9:18 PM, Karsten Hilbert wrote:

Sorry, I should have said just the line, and it didn't return anything.


OK, a bit strange, but then that might be due to Thonny.


Is Thonny an interpreter then.


It sort of is, or at least it runs one. We'd like to take
that out of the equation. I meant to run *just* an interpreter, namely,
the interactive shell built into Python itself.

IOW, run just "python" (or python3) on a command line and a shell
should open in which you can run the line in question.

(remember to define "time" appropriately)


From response just posted to Python-Tutor list:

WebRefs:
https://pythonprogramminglanguage.com/repl/
https://codewith.mu/en/tutorials/1.0/repl

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Aw: Re: stuck on time

[DL Neil via Python-list]

On 9/12/19 7:47 AM, RobH wrote:

On 08/12/2019 16:49, Dennis Lee Bieber wrote:

On Sun, 8 Dec 2019 09:44:54 +, RobH  declaimed the
following:

def print_time():
  current_time = time.strftime("%I:%M")

...


I don't know if that is the correct way as I am just using the code from
the project I am trying to do


This is showing a severe lack of understanding in how Python itself
operates, leading me to recommend rereading a Python tutorial book before
attempting to hack into some one else's code.


+1
(in other words, agreement with others who have suggested that your 
knowledge of Python and/or programming, is insufficient to cope with the 
project you've set yourself)




Err, excuse me, I was not attempting to hack into someone else's code.
As the code is in the public domain, I wanted it to work as is, like it 
did for the author, without changing anything.


Please don't be upset. There are multiple understandings of the words 
"hack" and "hacker". Whereas most 'computer people' take the word "hack" 
to mean exactly what you are doing (and "crack" to refer to illegal 
access or ill-intent); the sad state of journalism (and Hollywood) has 
resulted in a confusion of the two.


Within the Python community, the word "hack" is used freely and without 
rancour, and includes both the permitted use of someone else's code, as 
you describe; and improving said code, as per @wlfraed's detailed 
analysis of the code-base's short-comings, a few hours ago.




So why should I now start to learn how python works.


Only you can answer this: is the effort required to bend the code to 
your will, worth the pleasure or utility the result will bring?


Alternatively, look at how much time *volunteers* have thought 
worth-while investing in helping you! (kudos @Karsten)



There is a safety issue here too. What if the original author had wicked 
intent, and the code actually performs to your disadvantage. How would 
you know? The only way is to inspect the code - reading it for example.



Following-on from the "hacking" comment, if the original author offered 
it to you/the world, and uploaded this code to some public repo(sitory), 
(s)he will also be happy for you/others to correct and improve.


Added to the observation that the code is missing parentheses, it would 
seem that (this version of) the code would never work. Now that you have 
discovered this, the conventions of "open source" are that you will 
propose, or better still, supply a fix...



If the code doesn't work for me after a fair trial, I'll move on to 
something else.


Frankly, given @wlfraed's extensive analysis of the code, I'd be 
wondering about its utility too - but again, your assessment is the only 
one that counts.


(If I am confusing two recent list-threads, I apologise)
If you are going to be experimenting with IoT and/or SBCs, the sad 
reality is that it is not an area which has had time to 'collect' a body 
of mature software. Accordingly, lots of the code made 'available' for 
use will be in a fairly rough (or "early") stage. Thus, the greater 
*your* skill-set, the more likely will be success - same as for any 
hobby/work-project! Also, IIRC you are also short-cutting by using a 
Pi-Zero (designed for application and roll-out) rather than a board 
designed for experimentation - but I'm guessing, so again, please don't 
take offense.

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python3 - How do I import a class from another file

[DL Neil via Python-list]

On 9/12/19 7:29 AM, R.Wieser wrote:
...


Note that in all cases when you import a module (either by import the_file
or from the_file importe whatever) you actually import ALL of it


So much for my assumption only the class itself would be loaded - and a
wrench into my idea to have a number of classes in a "library" file.


There are three separate ideas which may be confusing:
1 we can 'load' a single class from a module, and
2 we can have 'library files' ("modules") which contains more than one 
importable-object

3 when a module is imported, parts of it are/may be executed at import-time

...


Update: While testing a bit more it turned out that, in the testcode, my
creating an instance with the same name as the class is to blame (who again
said that it isn't a good idea to use confusing names ?).  The moment I
changed the name of the instance everything works as expected.


and then there are the concepts of "scope" and "namespace".


Some of this you will know. (Hopefully) some will be 'new':


import moduleNM

- imports the entire contents of the module and executes what can be. 
This means that functions and classes are defined, and any executable 
code in the 'mainline', eg print( "Hello World" ); will be run.
- from that time you may then refer to the objects detailed within the 
module, eg


a = moduleNM.ClassA()

- note how the module has become part of the program, but occupies a 
separate "namespace"



from moduleNM import ClassA

- changes things so that you may now refer to ClassA without mention of 
the moduleNM, ie brings the name "ClassA" into the current namespace

- causes confusion if there is already a ClassA in the current namespace!


from moduleNM import ClassA as ClassA2

- also obviates the need to mention moduleNM/adds to the current namespace
- allows use of both ClassA-s - the 'local' ClassA as "ClassA", as well 
as providing an alias ("ClassA2") to the ClassA defined in moduleNM



import moduleNM as aliasNM; is also available!


WebRefs:
A comfortably readable expansion of the above: 
https://www.digitalocean.com/community/tutorials/how-to-import-modules-in-python-3
Python's view of (import-able) modules: 
https://docs.python.org/3/tutorial/modules.html
Other questions answered: 
https://docs.python.org/3/faq/programming.html?highlight=import, eg 
what-are-the-best-practices-for-using-import-in-a-module
The importlib library which under-pins "import": 
https://docs.python.org/3/reference/import.html
The full-fat range of import-possibilities: 
https://docs.python.org/3/library/modules.html--

Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: ModuleNotFoundError: No module named 'email.mime'; 'email' is not a package

[DL Neil via Python-list]

On 9/12/19 8:13 AM, b...@bbhoyer.com wrote:

Just registered
Thanks


Hi @bob, welcome to the gang...



  I am a beginner in Python, been working on class material from Mosh

...


  from email.mime.multipart import MIMEMultipart

...


  Here is the error message:
  Traceback (most recent call last):
    File "c:\Users\Owner\Desktop\HelloWorld\[5]email.py", line 1, in 

  from email.mime.multipart import MIMEMultipart
    File "c:\Users\Owner\Desktop\HelloWorld\[6]email.py", line 1, in 

  from email.mime.multipart import MIMEMultipart
  ModuleNotFoundError: No module named 'email.mime'; 'email' is not a 
package
  I have spent some time trying to figure out resolve ...
  Can you help me with this pistol of a problem …



("pistol"? ...he says, manfully-struggling with the temptation to 
suggest that you "make love not war"...)



Let's look at the information given (in the "stack trace":

<>>


On line 1, the code requests that a module named/addressed as 
"email.mime.multipart" be located ("found"), and an object 
("MIMEMultipart") be imported (etc, etc).


So, when executing line 1, Python was unable to find the specified 
module (let's over-simplify and use the word: "file").



Libraries from the Python Standard Library are not included in the basic 
"python" download, and have to be added/separately downloaded, when 
needed. I suspect this is the problem (but may not be)!



Sadly, I am not a user of MS-Win, so am loath to try to help much more, 
for fear of leading you along the wrong track.  Herewith some self-study 
which should put your boots (back) on the ground...



WebRefs: installing packages
This is more readable: 
https://protechguides.com/how-to-install-python-library/
This is from 'the book of words': 
https://packaging.python.org/tutorials/installing-packages/


NB I understand that "pip" is installed on MS-Win as part of python, so 
you don't need to worry about that/can quickly check. If your course has 
not taken you through "virtual environments" then feel free to ignore 
such, for now.

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Aw: Re: stuck on time

[DL Neil via Python-list]

It's a lot like the misuse of the word "theory".


You mean to say that in theory there is no difference between theory and 
practice, but in practice there is?


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: python 3 prefix to infix without too many parethesis

[DL Neil via Python-list]

On 10/12/19 8:40 AM, Terry Reedy wrote:

On 12/9/2019 6:21 AM, jezka...@gmail.com wrote:

Hi, I have got a problem.


Is this homework?


Same question - that way we know that our task is to help you learn to 
code in Python, cf a problem with Python itself...


Similarly, you may like to  know that there is a separate Python-Tutor 
Discussion List.




I wrote a code for prefix to infix. It works, but I need improve it
so on input there will be only necessary parentheses.


The example code processes, like a calculator (um, yes, well...). What 
do you mean by "prefix to infix"?




Define 'necessary'; give multiple input/output examples.
Put them in a test function.


+1



Here is the code:


Question: with abbreviated variableNMs, eg "p", "a", even "op"; how 
easily will you comprehend the code in six month's time? How easy do you 
think it is for someone else to read the code now? NB the print 
functions are no help because the labels are equally-abbreviated, plus 
my aged-eyes start to strain when following lists of %s-es and regexps - 
mea culpa!




import re
a = input()

class Calculator:

 def __init__ (self):
 self.stack = []

 def push (self, p):
 if p in ['+', '-', '*', '/', "**"]:
 op1 = self.stack.pop ()
 op2 = self.stack.pop ()
 self.stack.append ('(%s%s%s)' % (op1, p, op2))

 print("op1 =", op1)
 print("op2 =", op2)
 print("p=", p)
 print(self.stack)

 else:
 self.stack.append (p)


 def convert (self, l):
 l.reverse ()
 for e in l:
 self.push (e)
 return self.stack.pop ()

c = Calculator ()

print (c.convert (re.findall(r'\d+|\*\*|[-+*/]', a)))


input is like /-*+*++**85 27 39 87 65 65 37 63 91


ALWAYS, ALWAYS, ALWAYS, document a RegExp with a comment!!!

After 'translating' the abbreviations according to my recollection of 
this form of a frequently-used ComSc course problem, I was then 
surprised by the format of the input - using an infix calculator we 
enter a value, then an operator, and then another value, etc; and using 
a postfix calculator one usually inputs a couple of values and then an 
operator, another value, and then an operator, and so on - rather than 
receiving it all at once. Is this 'all at once' a feature of an infix 
calculator or am I missing/forgetting something?


For which I am obviously not the only one with a confused expression on 
my face...



Since '*' and '**' are both allowed operators (see 'p in' above), spaces 
are needed between operators to disambiguate.  In any case, what output 
does your code produce now, and what do you want it to produce.


I hate RegExps because it is so easy to write a wrong 'un. However, 
because the "\*\*" selection precedes the "[-+*/]", it appears (rheumy 
eyes allowing) to enact arithmetic priority correctly. However, as 
mentioned before, most of us will have created a serial and algorithmic 
approach to this problem, back-when; so it does seem a little strange.



OK, let's address your actual question: the parentheses!

How do you plan to make them work? As elsewhere: please provide example 
input and expected output, so that we can be sure of the code's 
objective. Multiple examples can only help, especially if there are 
exceptions and "edge-cases". Also, if your progress in Python is ready 
for it, we can talk PyTest (or equivalent), plus such a plan enables you 
to practice Test-Driven Development...



These problems stretch one's mental acuities. It would seem that I am 
not the only one looking forward to hearing back from you...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python3 - How do I import a class from another file

[DL Neil via Python-list]

It might be becoming 'long', but this discussion contains much of profit 
to many people!



Us 'silver surfers' do need to periodically question those beliefs we 
hold as 'tenets' of ComSc. Amongst them is RAM (or "core"!) 
conservation. It remains a virtue somewhat, but at the same time, 
storage costs are considerably cheaper than they were four decades ago, 
whereas the relative-cost of your time (to think about such things) has 
increased!


That said, many of our more youthful (and in their own words "better in 
every way", harrumph!) colleagues should also review their own beliefs 
from time-to-time, eg working with IoT devices certainly requires a more 
conservative and considerate approach to resources!
(and in such applications, we tend to enjoy significant advantage 
because we've 'been there, done that'!)



Similarly, it is a serious fallacy to attempt to directly 'translate' a 
'command' from one language into an 'equivalent' in another/a new 
language. Whilst the syntax may appear similar, such superficiality 
doesn't allow for subtleties of meaning! In the same way, if you ask one 
person to translate the English word "preserve" into German, and then 
another person to translate that German word into English, what are the 
chances that the 'answer' will be "preserve"? Given that "preserve" 
could mean a jam/jelly/conserve (noun) or (verb) 'maintain the meaning', 
with a similar range of synonyms (etc) on the German side; the word 
"preserve" is unlikely to be preserved!

(with all due apologies for such terrible humor and word-play!)


On 9/12/19 11:32 PM, Terry Reedy wrote:

On 12/9/2019 2:36 AM, R.Wieser wrote:

Terry,

Standard for in-file test is

[snip]

Any reason to put the testcode in a procedure instead of just have it
directly follow the "if __name__ == '__main__' " test ?


One can, for instance, interactively do

 >>> import mod
 >>> mod.test()

...

I went through this cycle of thinking, first using if __name__... to 
'hide' testing, then adding test functions within the module, and now 
keep test code quite separate from 'the real thing'.


In fact, PyTest (for example) will auto-magically look for test-code in 
sub-directories of the project directory, eg "projectNM/tests". So, for 
every (Python) file in my projectDIR, you should be able to find at 
least one corresponding Python test-file in the tests sub-dir!


This fits with your philosophy of keeping (imported) modules small, 
perhaps to the point of separating each class into its own module - if 
'efficiency' is the goal, why would you want to include test-code within 
a 'production system'?



On which point, if we had a class Person(); and then decided that in 
certain cases we wanted specific methods that didn't apply to every 
(type of) Person, we would likely create a sub-class, eg class Man( 
Person ); - although some might dispute the accuracy of such an "is a" 
relationship...


That being the case, were you to put the Man class' code into (say) a 
module called "man.py" and Person()'s code into "person.py", then a 
point made earlier is that you can't merely import man.py;! You must 
also ensure that person.py is imported, somewhere along the line. Thus, 
because it names (and requires the logic of) the super-class, the 
sub-class cannot be imported 'alone'. (regardless of saving-space 
objectives, etc)



Expanding on 'names' and del(): the first point is that there is seldom 
much to be gained by using del(), ie becoming your own 'garbage 
collector'. That said, there are some very specific times when it 
becomes absolutely necessary (otherwise the Python 'gods' would not have 
given us the functionality!). Once again, RAM is (relatively) plentiful, 
and when you need to replicate many multiples of an object, it is likely 
within a loop and therefore the same instanceNM is being 're-used' and 
the garbage-collector takes care of 'recycling' the storage space 
auto-magically.



Regarding 'names', yes some languages are "strongly typed" and (mostly) 
for that reason include an indicator of "type" within the name. Python 
enables you to play fast-and-loose, eg


>>> x = 1
>>> x = "abc"
>>> x = 2.3
>>> x = [ 1, 2, 3 ]
>>> x = ( 1, 2, 3 )
>>> x = 1, 2, 3

constitutes a perfectly legal program (where "legal" != "useful").


There again, there are times when values are processed into a different 
data structure, eg we start with a list of values, but only want to use 
unique values/process each representative-member once.


A Python-list will handle the former (wow, that's a bit of 'rocket 
science'!?) whereas a Python-set will only contain unique values. So, 
rather than using a somewhat-bland varNM such as "letters", I would 
choose to be more explicit:


>>> letters_list = [ "a", "b", "a", "c", "a", "d", "a" ]
>>> len( letters_list )
7
>>> letters_set = set( letters_list )
>>> len( letters_set )
4
>>> letters_set
{'b', 'a', 'd', 'c'}

There is no question whether "letters" is a list, or a set, (or ...), 
because

Re: More efficient/elegant branching

[DL Neil via Python-list]

I agree with you, so I'm going to ignore def branch2(a, b, z): and def 
branch3(a, b, z) because they appear too contrived - and I guess that's 
fair, in that you've contrived to satisfy pylint.


Question: are there other people/factors who/which should be regarded as 
more important than the linter's opinion?


The usual dictionary-controlled construct I've seen (and used) involves 
using functions as the dict's values:


def branch1( *args): pass
...
alternative[ x ] = branch1

and thus:

alternative[ choice ]( *args )


On 10/12/19 12:27 AM, Musbur wrote:
I have a function with a long if/elif chain that sets a couple of 
variables according to a bunch of test expressions, similar to function 
branch1() below. I never liked that approach much because it is clumsy 
and repetetive, and pylint thinks so as well. I've come up with two 
alternatives which I believe are less efficient due to the reasons given 
in the respective docstrings. Does anybody have a better idea?


def branch1(a, b, z):
     """Inelegant, unwieldy, and pylint complains
     about too many branches"""
     if a > 4 and b == 0:
     result = "first"
     elif len(z) < 2:
     result = "second"
     elif b + a == 10:
     result = "third"
     return result


Is it ironic that the language does not have a form of case/switch 
statement (despite many pleas and attempts to add it), yet the linter 
objects to an if-elif-else nesting???



One reason why this code looks a bit strange (and possibly why PyLint 
reacts?) is because it is trivial. When we look at the overall picture, 
the question becomes: what will the 'mainline' do with "result" (the 
returned value)? Immediately one suspects it will be fed into another 
decision, eg:


if result == "first": # do the first-thing...
if result == "second": # do the second-thing...

When found 'in real life', the above structure with such decisions is 
almost certainly followed by a second decision to select which 
'action(s)' will be carried-out. The original-code rarely stands-alone, 
because all we've achieved is a labelling of the circumstances according 
to the arguments provided.



Rather than using a variable for 'labelling', (all other things being 
equal) I'd prefer to go straight to a function which 'does the business' 
- and use the function's name as a documenting 'label'!



A quick reflection shows that I have vacillated between two approaches:

1 put the decision-making in the 'mainline' and call a variety of 
functions/methods to do 'the first thing' etc. ie more-or-less what you 
have here, except the cascade of decisions is not separated from 'what 
came before'.


def calculation( value, operator, operand):
if operator == "+":
add( value, operand)
elif operator == "-":
sub( value, operand)


2 arrange/call a series of functions each dealing with a single 
circumstance, eg (borrowing from @Chris) should I "tap"?. Thus each 
function will firstly check if the circumstances demand that action, ie 
the applicable one of the if-clauses above, and if they do, then 
implement that action: 'do the first thing' (etc).


value = handle_add( value, operator, operand )
value = handle_sub( value, operator, operand )

where:

def handle_add( value, operator, operand):
if operator == "+":
return value + operand
return value

Ugh - the example is no less contrived and grossly artificial...


The second of those appears somewhat 'cleaner' - in that we have 'hidden 
away' all those pesky if-constructs; BUT it is NOT immediately obvious 
exactly how many of the series of functions will actually result in any 
'action' (if not, all of them). So, perhaps that raises the 
(earlier/elsewhere) question of whether the various action-functions are 
dependent or independent/exclusive of each other? It also requires 
carefully choosing descriptive function names!

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: distinguishable matplotlib colours / symbols / line styles

[DL Neil via Python-list]

On 17/12/19 5:19 am, Chris Angelico wrote:

On Tue, Dec 17, 2019 at 3:16 AM duncan smith  wrote:


Hello,
   Not really specific to Python or matplotlib (but that's what I'm
using). I'm looking for a good combination of colours and symbols for
scatter plots, and combination of colours and line styles for line
plots. Too often I find myself producing these when I don't know whether
they will end up being printed in colour or greyscale. It would be a lot
easier if I could produce decent plots that would work well in either
case. I can find various stuff on colour palettes, but pretty much
nothing on symbols or line styles. Is anybody aware of an approach that
works well? I'm aware of issues with colour blindness and RGB versus
CMYK. Ideally I'd have something that I could just use that would deal
with all these issues. TIA.



I'd recommend looking at the Web Content Accessibility Guidelines
published by the W3C; there are a number of tools out there that are
designed to help you pick colours for a web page, and the same sorts
of rules will apply here, I think.

Also, thank you for even *thinking* about this. A lot of people don't. :)


+1

We spend a lot of time teaching this topic (non-Python courses). It 
receives a lot of often highly-polarised comments/discussion. Many folk 
have their 'eyes opened' to an issue which has not affected them 
personally. Some even have to be informed that it is a legal obligation 
in their jurisdiction. However, it also receives the highest number of 
'why do I have to learn this stuff' complaints...


I learned (way back) that the incidence of "color blindness" is far 
higher than I had imagined. Secondly, that it affects males more than 
females. Thirdly, that calling it "blindness" is a bit of a misnomer, 
because whilst people often can't see red 'correctly' (most common 
symptom), they do see something (it varies). Which is why they are 
permitted to drive vehicles (traffic lights: red, amber/yellow, green - 
and arrows; plus stop/brake lights), but why many smart-phone apps/web 
pages which encode information-relevance (red is 'wrong' and green/blue 
is acceptable) can become almost unusable (without other cues).


Those key-words: "accessibility guidelines" will yield a swag of useful 
tools - ignore the ones which are basically 'help choose the color of my 
web page/color palette, because they are often aiming (only) for 
'pretty'. The best tools enumerate the efficacy of fg/bg 
color-combinations, allowing one to experiment; and will enumerate 
grey-scale variation/comparisons.



Hmmm, note to self (you've inspired me to specifically review/critique 
the printing-from-screen action): what happens when we take a 
color-checked screen display and print same but end-up viewing it as 
monochrome/grey-scale output? Probably not a main-stream demand, but 
worth tossing at the WCAG experts...

--
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python3 - How do I import a class from another file

[DL Neil via Python-list]

Wow, I turned my back to attend to $job and this conversation has 
'exploded' (>80 msgs)!


TLDR;
With apologies, I started to reply to your reply, but then added 'bits' 
as I read the conversation thereafter. The result (below) is a messy 
hodge-podge, for which I can only apologise (as I don't have more 
spare-time to be able to 'fix it' and edit it into some semblance of 
proper order). May I suggest a first quick-read to pick-up the 
'highlights', and then re-reading in order to make sense of the 
cross-references? I regret the appearance of circling-back between topics...




On 10/12/19 9:45 pm, R.Wieser wrote:
...


Than again, I normally camel-case my variable and function names (to make
them stand out from the build-in commands), as I normally do not use editors
with syntax highlighting.

...


:-) Always start with a descriptive* name for the variable, regardless of if
it is a compounded one or not.   As said, the type prefix is just a
reminder.


Remember that PEP-8 is (only?) applied to code submitted for inclusion 
in the Python Standard Library. It's wider application is less 
prescriptive (although many in-house style manuals take an 
embrace-and-extend approach).


Bottom-line (IMHO):
- if you are coding by yourself and only for yourself, you make your own 
'rules'!
- if you are working with others, you (all) come up with 'the rules' - 
one of which may be that you allow 'the other guy' to make his own 
rules, whilst (s)he does the same for you - and simply work-around that 
with your own (and, one assumes, v-v)
- if the organisation/team already has 'rules', then you adapt to the 
group mores...


Have you come across the bit where Guido (I think?) talks about "the 
hobgoblin" of over-applying 'rules'? To which the 'Zen of Python' adds 
(the joke) "unless you're Dutch".


The topic of naming-standards is a massive time-sink/hobgoblin (see also 
- using tabs or spaces to indent, and 'how many spaces')...
(see also: which car is 'best', which language is 'best', which team..., 
which super-hero..., do you really love me? does my bum look big in this...)


That said, there's enough to learn when taking-on a new language, 
without 'sweating the small stuff' ("cognitive load"). So, if it helps 
you to use VS Coding Standards right now, just do it. Later, as you come 
to understand more of the 'philosophies' behind Python, you will/may 
learn/adapt (see also, 'working with others', above).




*though not /too/ descriptive.   Languages which can easily have
40-character names for variables, functions and such are simply the pits,
especially when they start to differ only for a single/few character(s)
somewhere in the middle. :-(


Surely it is unlikely that all 40-characters are necessary to 
communicate meaning!? Even in Welsh?Tamil?Dutch?German?


That said, it is *so* easy with modern text editors to make universal 
changes (an error-laden task filled with so much peril it was 
strenuously-avoided in the ?good, old, days) - which ease applies as 
much to variableNMs as it does to interpreting what tab-to-indent 
means/is to mean for this module!


In other words, I play as fast-and-loose as does Python (per earlier 
comment).




rudy = Man( etc ) #rudy probably does have class, in this instance

You almost owed me a new keyboard there. :-p


Normally my lame jokes are greeted with a groan, so I'll take that as a 
compliment (regardless of how it was intended - I'm that desperate!)



Which brings me to a few points, noted throughout the conversation. 
Whereas I do offer training (not Python), hold a qualification in the 
field (specifically vocational training/andragogy), and have a research 
interest in Cognitive Psychology (yeah, yeah, blah, blah, big deal...) 
there will be very few others, similar, who are also (active) members of 
this list. Expecting high-quality 'training materials' as responses is 
probably asking 'a bit much'...



We do have folk (on the list) who are actively developing the language, 
others who use the language professionally and every-day, and still 
more, less specialised, right ?down to 'mere' hobbyists who only get to 
talk-Python at (some) weekends. Answers reflect that. However, a scan of 
(names frequently appearing in) the archives will soon reveal 'much'!


In case of misunderstanding:
There is no-one (to my knowledge) who is paid to be 'here'. All 
contributions are voluntary and provided out of people's 'free time'. It 
is not a manufacturer's 'support site'!


The best way to receive help with coding (as has been said) is to 
copy-paste *your* problematic code into a post. This allows interested 
parties to easily reproduce the problem (or not). It also ensures that 
responses are as specific as possible by providing 'focus'!


It also pays to continue the copy-paste-result pattern when replying to 
advice: 'I've tried it, and 'this' is what happened' - yet I understood 
you to say 'that' should...' Such not only ensures/contributes to 
ensuring that 

Re: distinguishable matplotlib colours / symbols / line styles

[DL Neil via Python-list]

On 17/12/19 3:37 pm, Dennis Lee Bieber wrote:

If, by some chance, external nodes can get to it:
http://wlfraed.microdiversity.freeddns.org/BW/BWConv.html }


Works for me, no hacking necessary!
(photo of ppl dressed-up in Mickey Mouse type costumes)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to combine a group of lists together

[DL Neil via Python-list]

On 21/12/19 6:04 pm, wanghao7...@gmail.com wrote:

I utilized .get_text() + .split() functions and obtain this:
spe=
['$278.86as', 'of', 'Dec', '20,', '2019,', '06:47', 'PST', '-', 'Details']
['4.7', 'inches']
['750', 'x', '1334']
['64', 'GB']
['2', 'GB']
['Apple', 'A11', 'Bionic,', 'Hexa-Core,', '(2x', 'Monsoon', '+', '4x', 
'Mistral)']
['5.22', 'ounces']
['No']
['No']
['12', 'MP']
['7', 'MP']
['1821', 'mah']
['No']
['Yes,', 'on', 'the', 'front']
['None']
['iOS', '(Last', 'OS', 'info:', 'IOS', '11)']
['September', '2017']

They have a shared name 'spe', but no concrete names for each list.
If I use spe[0], I can only get all the first element of each list.

How can I mix all of these elements into a single list like:
['$278.86as', 'of', 'Dec', '20,', '2019,', '06:47', 'PST', '-', 
'Details','4.7', 'inches'..]


What is type( spe )? Please copy-paste code/session into your email msg 
- the above is NOT Python.


Your best solution may be to use a loop (instead of the constant 0, as 
list-argument).


The answer to your specific question is 'in the manual': list.extend()
(but a loop will likely still be required...)


WebRef:
https://docs.python.org/3/library/stdtypes.html#sequence-types-list-tuple-range


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to extend an object?

[DL Neil via Python-list]

On 21/12/19 2:50 pm, Greg Ewing wrote:

On 21/12/19 1:59 am, Stefan Ram wrote:

   I would like to add a method to a string.

   This is not possible in Python?


It's not possible. Built-in classes can't have methods added
to them.

You can define your own subclass of str and give it whatever
methods you want.

But in your case:


for s in 'abc'.like( '(.)' ):


I would recommend making it a stand-alone function instead,
so that you would write

   for s in like('abc', '(.)'):



Contrarily: sub-classing str and making like() a method strikes me as 
the way to go, particularly if the 'abc' disappears into the instance's 
__init__() - and better still if the '(.)' may also be absorbed as a 
class/instance attribute, and even more-so if like() is not the only 
method/stand-alone function required for this category of strings...


but...

choose a self-documenting name for the class, and
choose a better-documenting name for the method.

instance = class( "abc" )
...
for s in instance.method():

is clean, self-documenting (or will be once you've done your thing), 
easier to test (fewer args => easier assured testing).


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: More efficient/elegant branching

[DL Neil via Python-list]

On 11/12/19 1:07 AM, Daniel Haude wrote:

Hello Neil,
thanks for the detailed answer.


Question: are there other people/factors who/which should be regarded
as more important than the linter's opinion?

Yes. Mine.


Um, see below...
(unless humor)


I was just puzzled at the linter's output (took me a while to figure out 
what it actually meant), and that got me started on the track if there 
was maybe a more Pythonic way of dealing with that decision chain.

...


One reason why this code looks a bit strange (and possibly why PyLint
reacts?) is because it is trivial. When we look at the overall
picture, the question becomes: what will the 'mainline' do with
"result" (the returned value)? Immediately one suspects it will be fed
into another decision, eg:


No, the "result" is a text message that actually means something and is 
eventually output for human consumption.


The whole thing is rather academic. Also my efficiency argument doesn't 
hold water because this routine is executed just a few times per hour. I 
like the "condition table" approach for its lower line count but I'll 
stick with if/elif because it's more conventional and therefore easier 
to understand for the casual reader.


+1 (except that it's your future-self or another maintainer who is 
important rather than anyone 'casual')



A reason we are having difficulty making suggestions is because the 
example has been abstracted. Thus we had no idea that the objective was 
to (possibly) output to SMS, nor more importantly, the meanings of a, b, 
and z. (it's all algebra to me!)


In some/another application you may like to consider constructing the 
source-data for this decision within a class. Then the conditions can 
become methods or even properties of the class - remembering that a 
property could be some combination of attributes, eg a > 4 and b == 0. 
Alternatively, the class could maintain some idea of 'status', where 
"first" corresponds to the preceding 'state', "second" when len(z) falls 
below 2, etc. The class's property/ies thus simplify the 
SMS-transmission process/class...


Appropriate name-choices will make the code self-documenting - per the 
'reading' objective:


if employee.worked_overtime:
sms.send( "We're in the money...!" )

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Most elegant way to do something N times

[DL Neil via Python-list]

On 24/12/19 1:04 PM, Chris Angelico wrote:

On Tue, Dec 24, 2019 at 10:45 AM Marco Sulla  wrote:




I encounter with cases like doing a function 6 time with no argument, or
same arguments over and over or doing some structral thing N times and I
dont know how elegant I can express that to the code.



??? Excuse me, but why you needed to call the same function SIX times? This
seems to me not elegant in primis.

Can you give us a practical example?


Taking the top/bottom six from a sorted list of occurrences.

eg in Accounts Payable, a clerk might be given the task of 'chasing' the 
six largest debtors (people who owe us money), as today's work 
assignment/priority.




File parsing. You read a section header and want to ignore that
section, so you ignore the next 15 lines.


(just to be cheeky to @Chris)

Perhaps better as a Finite State Machine, with one state being 'seeking 
section header'.

1 can we guarantee that the 'magic constant' of 15 will always apply?
2 presumably the total routine will involve more than identifying a 
single header and skipping (only) that section.

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lists And Missing Commas

[DL Neil via Python-list]

On 24/12/19 1:48 PM, Tim Daneliuk wrote:

If I do this:

 foo = [ "bar", "baz" "slop", "crud" ]

Python silently accepts that and makes the middle term "bazslop".

BUT, if I do this:

 foo = [ "bar", "baz" 1, "crud" ]

or this:

 foo = [ "bar", 2 1, "crud" ]

The interpreter throws a syntax error.

This is more of an intellectual curiosity than anything else, but why do 
strings silently
concatenate like that, while all other case blow up with an error.  i.e., What 
is about
the language the promotes this behavior.  At first blush, it seems 
inconsistent, but what
do I know ...


Restricting our conversation to numbers/integers and strings, 
"concatenate" only ever seems to apply to string objects.

(there are meanings of "concatenate" for matrices, for example)

For fun, try:

help( str ) # and
help( int )

They both mention __add__(self, value, /) In both cases such is defined 
as "self+value" - but remember that the "+" operator is "over-loaded". 
Thus (and I'm telling you nothing new here) when dealing with strings we 
read "+" as "concatenate"; whereas with integers it is plainly "add". 
Two quite different operators/operations!



However, your point involves the fact that whereas:

1 + 2   # 3 is *clearly* addition, and
"a" + "b"   # "ab" is *clearly* concatenation

"a" "b" # also evaluates to "ab"

and is thus, concatenation without any explicit infix operator! Just one 
cotton-picking minute - what's going on here?


The answer to all your dreams (um, maybe...) lies in "The Python 
Language Reference" docs, particularly Section 2 "Lexical Analysis" (the 
process of taking "tokens" and making sense of how they go-together!).


<<<
2.4.2. String literal concatenation
Multiple adjacent string or bytes literals (delimited by whitespace), 
possibly using different quoting conventions, are allowed, and their 
meaning is the same as their concatenation. Thus, "hello" 'world' is 
equivalent to "helloworld". This feature can be used to reduce the 
number of backslashes needed, to split long strings conveniently across 
long lines, or even to add comments to parts of strings, for example:


re.compile("[A-Za-z_]"   # letter or underscore
   "[A-Za-z0-9_]*"   # letter, digit or underscore
  )

Note that this feature is defined at the syntactical level, but 
implemented at compile time. The ‘+’ operator must be used to 
concatenate string expressions at run time. Also note that literal 
concatenation can use different quoting styles for each component (even 
mixing raw strings and triple quoted strings), and formatted string 
literals may be concatenated with plain string literals.

>>>


Thus, the adjacency of two literals explicitly only implies 
concatenation for strings. There is no equivalent/similar mention for 
numbers.



WebRef: https://docs.python.org/3/reference/lexical_analysis.html
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lists And Missing Commas

[DL Neil via Python-list]

On 24/12/19 3:35 PM, Chris Angelico wrote:

On Tue, Dec 24, 2019 at 12:56 PM DL Neil via Python-list
 wrote:

However, your point involves the fact that whereas:

1 + 2   # 3 is *clearly* addition, and
"a" + "b"   # "ab" is *clearly* concatenation

"a" "b" # also evaluates to "ab"

and is thus, concatenation without any explicit infix operator! Just one
cotton-picking minute - what's going on here?


1_2# Clearly concatenation. Right?
0_0# Clearly an emoticon

Just another cotton-picking minute..



Shows what you know: 1_2 is marching orders

- and here are yours...
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Lists And Missing Commas

[DL Neil via Python-list]

On 24/12/19 5:20 PM, Tim Daneliuk wrote:

On 12/23/19 7:52 PM, DL Neil wrote:


WebRef: https://docs.python.org/3/reference/lexical_analysis.html



Yep, that explains it, but it still feels non-regular to me.  From a pointy 
headed academic
POV, I'd like to see behavior consistent across types. Again ... what do I know?



I thought it was 'the boss' who has the "pointy head'? (so that any 
'difficult' idea will not land, but simply slide-off to one side or the 
other?)


It is an interesting lateral-thinking enquiry, and we should not simply 
accept 'stuff', but question how it best works/we can best use it.



That said, on the surface, I might agree.

Venturing into compiler-writing/lexical analysis, and comparing with 
other languages, we rapidly realise that it is 'odd' (or special).



However, keep reading (the above web.ref) and find the section where 
"white space" is described. Combine that with the idea/nuisance-value of 
splitting long strings over multiple lines.


Rather than puzzling-over an 'only/special/weird concatenation' 
allowance, perhaps we should see a 'convenience factor'?

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: [Q] __init__ in class

[DL Neil via Python-list]

On 27/12/19 12:11 AM, 황병희 wrote:

in making class, why we should set "__init__"? so non-init class is
useless? actually i did fail to understand __init__'s meaning. what is
role of __init__ in class.

sincerely, from python beginner.



Please be aware that there is a Python Tutor Discussion List.

If you are a beginner in Object-Oriented Programming, Wikipedia offers a 
multi-language, if dense introduction. This is complemented by their 
Class (computer programming) page.


If you are new to Python's implementation then don't go past the Python 
Tutorial (in particular Section 9).


There are plenty of web-tutorials discussion Python classes. 
HackerEarth's aligns nicely with the Wikipedia definitions (or the 
abstract terminology introduced in many ComSc texts and courses).



WebRefs:
Python Tutor: https://mail.python.org/mailman/listinfo/tutor
Object-oriented programming: 
https://en.wikipedia.org/wiki/Object-oriented_programming
Class (computer programming): 
https://en.wikipedia.org/wiki/Class_(computer_programming)

The Python Tutorial: https://docs.python.org/3/tutorial/index.html
HackerEarth: 
https://www.hackerearth.com/practice/python/object-oriented-programming/classes-and-objects-i/tutorial/

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: [Q] __init__ in class

[DL Neil via Python-list]

On 27/12/19 10:58 AM, Irv Kalb wrote:

On Dec 26, 2019, at 3:11 AM, 황병희  wrote:
in making class, why we should set "__init__"? so non-init class is
useless? actually i did fail to understand __init__'s meaning. what is
role of __init__ in class.


...


As an example, let's say that you create a point class, where the Point has 
both an x and a yY value.  Your class might look like this:

class Point:
 def __init__(self, x, y):   # When you create a Point object, you must 
pass in a value for x and y
 self.x = x
 self.y = y

 #Then you would probably have other methods like:

 def getX(self):
 return self.x

  def getY(self):
  return self.y

  def show(self):
   print('The values of this point are:', self.x, self.y)

etc.

Given a class like that, when you create an object from the Point class, you 
pass in values for x and y.  The code in the __init__() method saves away those 
values into instance variables.  Those instance variables can be used by other 
methods.  In this example class, getX, getY, and show.

Here is some sample code to create two Point objects, then ask them to show 
their contents:

pointA = Point(10, 15)
pointB = Point(-3, 38)
pointA.show()
printB.show()



Is this a/the pythonic way to access attributes?

It appears Java-like, or some other language transliterated into Python. 
Further, the naming convention suffers similar heritage.



What was not shown is that in order to access the values of Point()'s x 
and y attribute, the following is needed:


x_value = pointA.getX()

which in and of itself is not too bad, if wordy and ugly; but what 
happens when we perform a vector addition?


point_a_plus_b = Point( pointA.getX() + pointB.getX(),
pointA.getY() + pointB.getY()
)

Which is still not too shocking, but imagine a more complicated 
calculation, eg widen to include z- and w- dimensions or consider 
n-dimensional matrices...
(there are optimised tools for such, but continuing with the post-er's 
example)



Rather than a 'closed' view of objects ("encapsulation") Python 
maintains a more 'open' approach. "Getter" and "Setter" functions are 
put-aside in favor of direct-access:


value = instance.attribute  # 'getter'
instance.attribute = value  # 'setter'
del instance.attribute  # 'destructor'


Now, imagine a class with ten attributes (for example) in Java (and 
similar). Before use, we would have to write ten getter-functions, ten 
setter-functions, and maybe (just to be 'complete') ten 
destructor-functions. Whereas in Python, nothing!


Whatever you do, don't mention this to such coder-colleagues! We are 
able to code so much faster than they, and are considerably less 
restricted by our choice of language, thus promoting numerous efficacies!



There are times when we can't allow 'just any value' to be applied to an 
attribute. For example, should a person's birth-date be recorded as 
ten-years from now? So, when we need to check or 'massage' data before 
use/recording, we might turn the attribute into a "property". This does 
give us the Python (and pythonic) version of getters-and-setters.


Similarly, there are tactics which do enable data-hiding (or strict 
"encapsulation") for those occasions when it is needed...but now we're 
likely way beyond the OP's needs!



WebRefs: (additional to those previously-posted in this thread)
https://www.python-course.eu/python3_properties.php
https://www.python.org/dev/peps/pep-0008/
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Friday Finking: Source code organisation

[DL Neil via Python-list]

Is it helpful to, and thus, do you have a style/convention for ordering 
the methods within each class in your code?



Python's "honking great idea" of namespaces enables us to gather 
functions/methods under a single name-umbrella, thereby avoiding 
name-clashes/name-space 'pollution'. [Oh yeah!]


Thus, if we collect a bunch of module-functions, we refer to them 
collectively by module-name/import-name, eg


import collected_functions_module as cfm
...
cfm.make_it_happen()
cfm.make_it_better()

Similarly, methods within a class may be accessed either as 
class-methods, or through instances (as required).

[am assuming no code example necessary]

A major difference however, is that if our mythical collection of 
module-functions has an internal-reference, eg b() requires a(), then 
function a() MUST exist, ie be defined, 'before' function b(). Whereas a 
class's methods may be defined in any (complete) sequence.


In the ?good? old days, eg COBOL - or more logically: when source code 
was most-usually viewed as printouts and the prevailing coding-style was 
for largely monolithic program(me)s; we would often number subroutines 
(initially in tens) to better navigate long listings of source-code, and 
more easily locate the subroutine's code from some reference-point. In 
data-terms, we might even create an abbreviation of a record's name and 
use that as an hyphenated-prefix for all of its data-fields' names.


Today, such artifice is unnecessary, given that code-editors (normally) 
offer one-click access from reference to source - even spanning multiple 
Python modules. [another, "oh yeah"!]



So, do you have an orderly method [hah!] for presenting/locating 
class-methods (and module-functions) within your code?


- why bother, the editor does 'the heavy lifting'
- dunders to the fore
- alphanumeric sequence by name
- order of appearance/use in 'mainline code'
- as they sprang to mind during TDD-creation
- most-used first, least-used last
- my code 'at the top', their stuff later...
- names of Monty Python characters by TV appearance date
or,
- some combination of ideas
and,
- how do you vary the above when dependencies intrude?

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Finking: Source code organisation

[DL Neil via Python-list]

"Define before use" is a broad principle that I try to follow, even
when the code itself doesn't mandate this.


IMO it makes the code easier to navigate even when it's not strictly
necessary. As others have said, Python mandates that the functions be
defined before they're CALLED, but I find that it's worth being
stricter. As a general rule, any global name in my code (that includes
constants, module-level functions, etc, etc) will have its definition
as the textually-first instance of that name. Or, at worst, the first
instance will be a comment immediately above the definition.


('Anonymised' because this is not 'aimed' at any particular individual)


I was somewhat amused at being 'corrected' in the "defined" cf "called" 
discussion. The 'corrections' are more precise. However such 
distinctions are machine-related, ie how does the Python interpreter behave?



The central topic of this (ideally, thought-provoking) discussion, is on 
the person (not the machine). Yes, the author. Although, if you 
subscribe to the philosophy that program(me)s should be written with 
human-readers in-mind, then the likes of "beauty is in the eye of the 
beholder" and "readability counts", apply!


Various 'gurus', probably even before the Agile/Gang-of-Four/OOP, have 
talked of code craftsmanship having a number of characteristics. One of 
which is that *my* code should appear pretty-much as *you* expect the 
solution. When we were asked to discuss this, over-the-teacups (on 
Friday, hence the "FF" moniker) the young fellow's question was phrased 
"what do you guys expect [from me]".
(yes, he was quoting/abstracting from 'Uncle Bob' and "Clean Code" 
[Prentice-Hall, 2009]).


We have a number of 'what do you expect'-s in Python. For example, there 
is no "law" saying that we must use "self" within classes. Other 
languages use similar but alternate terminology, eg "this". I could 
decide to use "dn_was_here" and the interpreter will be perfectly happy 
- but because you don't expect it and would be made to (unnecessarily) 
comprehend, I doubt it would promote our friendship/mutual-respect!


So, do you hold any 'expectation' relating to the organisation of class 
methods?

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Finking: Source code organisation

[DL Neil via Python-list]

On 29/12/19 2:19 PM, Tim Chase wrote:

On 2019-12-29 12:52, Greg Ewing wrote:

I tend to do this too, although it's probably just a habit
carried over from languages such as Pascal and C where you
have to go out of your way to get things in a different
order.


Apparently I'm not alone in my Pascal/C-derived habits of
define-before-use.


I didn't expect to be the only one who would bring 'prior experience' 
into it...




Inside a class, I tend to roughly follow
   __new__ (if present)
   __init__
   other dunder methods
   subsequent methods alphabetically


+1

Should __new__() and __init__() break with convention because its/their 
method/function signature is (really) part of the class's signature, and 
thus there's a need for proximity?

(aka is "expected")
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Grepping words for match in a file

[DL Neil via Python-list]

On 29/12/19 5:14 AM, Dan Sommers wrote:

On 12/28/19 12:29 AM, Mahmood Naderan via Python-list wrote:

Hi
I have some lines in a text file like
ADD R1, R2
ADD3 R4, R5, R6
ADD.MOV R1, R2, [0x10]
If I grep words with this code
for line in fp:
 if my_word in line:
Then if my_word is "ADD", I get 3 matches. However, if I grep word 
with this code

for line in fp:
 for word in line.split():
 if my_word == word:
Then I get only one match which is ADD R1. R2.
Actually I want to get 2 matches. ADD R1, R2 and ADD.MOV R1, R2, 
[0x10] because these two lines are actually "ADD" instructions. 
However, "ADD3" is something else.

How can I fix the code for that purpose?

(1) word.startswith() won't solve your problem.  Expliticly
checking the character after the second "D" would.  You'll
have to determine which characters are or aren't part of
the instruction.  A complete solution probably depends on
what else you are or will look for in the future.

(2) That looks like a programming language (88000? POWER?).
Watch out for comments containing the word ADD, too.



For which reason, and given its finite and limited nature, I'd prefer to 
be explicit:-


Assuming "ADD" is only a sample of the total task, perhaps it will not 
be too impractical to create categories of instruction:


ADD_CATEGORY = [ "ADD", "ADD.MOV" ]
etc

Now, you can perform an "in" on the first token:

if line.split()[ 0 ] in ADD_CATEGORY:
# this command one of those in the ADD category...

In this way, rather than relying upon (actually *not* being able to rely 
upon) similarities of appearance, you have complete control to limit 
which commands fit into which category/ies...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Finking: Source code organisation

[DL Neil via Python-list]

On 29/12/19 5:49 PM, Cameron Simpson wrote:

On 29Dec2019 09:49, Chris Angelico  wrote:

"Define before use" is a broad principle that I try to follow, even
when the code itself doesn't mandate this. This generally means that
"if name is main" is the very last thing in the file, and if there's a
main() function or equivalent, that's usually just before that. Any
metaprogramming goes right at the top; sometimes this is mandated (if
I write a decorator function, it has to be above the functions it's
decorating), but even if it's not, metaprogramming goes before the
mainline.


For main, i have the opposite habit. If a module has a main() function 
for command line use I usually want that right up the front:


  #!/usr/bin/env python3
  
  import...

  def main(argv=None):
    ... main command line ...

  classes, functions, etc

  if __name__ == '__main__':
    sys.exit(main(sys.argv))

My reasoning here is that I want the main programme obvious up front.

But then I loosely follow "define before use" after that.



OK, I'll bite:

1 the reader (one assumes) starts at the top, then scrolls all the way 
to the bottom to find if...__main___, but is then directed to search for 
the def main... which is all the way back up to the top!


2 if instead of a main(), that code was under if ... __main__ would that 
be an equally reasonable and "obvious" place to find it?


--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Friday Finking: Source code organisation

[DL Neil via Python-list]

On 3/01/20 7:36 AM, Terry Reedy wrote:

On 1/2/2020 2:46 AM, Cameron Simpson wrote:


Inline code under the if...__main__ stuff cannot be called as a function;


Which makes it difficult to test app startup.

I usually consider the main() function a reusable component. 


Which, if called (at least optionally) with a list of string arguments 
(sys.argv) as Cameron suggested, makes it easy to test with various 
lists of strings.



NB I agree with suggestion that 'important stuff' is better placed at 
'top' of source-code. Preceding comment assumes that main() is to be a 
word/name which catches the eye and has 'importance' attached. However, 
the Python idiom is "if __name__ == '__main__'" and (paradoxically?) 
that usually goes at the 'bottom'.


I think we've previously discussed if __name__ == '__main__' and/vs main().


Continuing the discussion, above:-


def start_up( args ): etc

def do_it( args ): etc

def close_down( args ): etc

if __name__ == '__main__':
   start_up( sys.argv )
   do_it( etc )
   close_down( etc )


1. app startup remains test-able
2. each component is reusable
3. sys.argv may be passed with all flexibility
4. separation of concerns, etc, observed

NB above for casual illustration only - if start_up() is more than 
trivial, seems likely that others would be better implemented as context 
manager, etc, etc...

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: heap enhancements

[DL Neil via Python-list]

On 4/01/20 2:13 PM, rontoto wrote:

Hi guys, the code is working, but I want know, what I can improve on this code. 
It is like implementation of the simulation of shipment clearance in the depot 
according to priority and available deliveries. How can I do it better? Can u 
show me? Thnx for your time.



1 add comments which document the code, explaining what each method 
accomplishes, etc


2 choose meaningful names - it is not like you're sending a telegram 
(and if you don't know what on earth I'm talking about, then this 
applies double!) and being charged by the letter.


Have you run any time-trials to establish whether the code is 'fast' or 
'slow'?


Recommended reading:
1 supporting the above, including PEP-8
2 supporting the above
3 writing program(me)s so that others can read them
4 search for "Python heap"
5 download and inspect heapq from the PSL (Python Standard Library)

There are algorithm-mavens on-list who might weigh-in at that level...
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python, Be Bold!

[DL Neil via Python-list]

On 7/01/20 6:10 AM, Abdur-Rahmaan Janhangeer wrote:

On Mon, 6 Jan 2020, 20:50 Rhodri James,  wrote:
No if you read the thread (please do it), that's the 4th time i'm
saying it's a bad idea



I have not (been following the thread) - with all due apologies.

Podcast of possible interest:
Episode #245: Python packaging landscape in 2020
https://talkpython.fm/episodes/show/245/python-packaging-landscape-in-2020
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list