[Tutor] Obfuscated Python [was Long Lines techniques]
On Thu, Dec 13, 2018 at 11:07:59PM -0500, Avi Gross wrote: > Python may claim to be straightforward but I can easily see ways > to fool people in python too with dunder methods or function closures or > decorators or ... Dunder methods shouldn't fool anyone. Each dunder method has a straightforward use, and most of the time you can guess that they do. __add__ implements the + operator, etc. Closures and decorators can be abused, as can anything. One can write obfuscated code in any language. Here's some amusing obfuscated Python: https://www.reddit.com/r/Python/comments/i1qgp/reduce_and_lambda_two_great_tastes_that_taste/ https://mail.python.org/pipermail/python-list/2013-October/658470.html Anyone else want to share some obfuscated Python code? -- Steve ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Long Lines techniques
On Thu, Dec 13, 2018 at 11:07:59PM -0500, Avi Gross wrote: [...] > There are cases where it may make sense to have a long like connected by AND > or OR given how python does short-circuiting while returning the last thing > or two it touched instead of an actual True/False. For example, you may want > to take the first available queue that is not empty with something like > this: > > Using = A or B or C or ... or Z > Handling = Using.pop() > > Sure, that could be rewritten into multiple lines. using = (A or B or C or D or E or F) [...] > I recently wrote some code and ran into error messages on lines I was trying > to keep short: > > A = 'text" > A += "more text" > A += object > A+= ... Without knowing the error message, its impossible to say what the problem is. My guess is that the object on line three wasn't a string. In which case, the obvious fix is: A += str(object) -- Steve ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Long Lines techniques
On Fri, Dec 14, 2018 at 01:03:55AM +, Alan Gauld via Tutor wrote:
> I'd probably suggest
>
> stgs = ''.join([
> "long string",
> "another string",
> ...
> "last string"
> ])
That's certainly better than using the + operator, as that will be quite
inefficient for large numbers of strings. But it still does unnecessary
work at runtime that could be done at compile time.
Hence my preferred solution is implicit string concatenation:
stgs = ("long string"
"another string"
"last string")
Note the lack of commas.
[...]
> > or even generator expression.
>
> These are usually just parens in a function. The generator
> expression is the bit inside (not including) the parens.
Hmmm, well, yes, no, maybe. The definition of a generator comprehension
*requires* it to be surrounded by parentheses. You cannot write this:
gen = x + 1 for x in items
you must use brackets of some form:
- [] for a list comprehension;
- () for a generator comprehension;
- {} for a dict or set comprehension;
But for the generator case, a short-cut is allowed. If the expression is
already surrounded by parens, as in a one-argument function call, you
can forego the inner brackets. So instead of:
flag = any((condition(x) for x in items))
we can just write:
flag = any(condition(x) for x in items)
--
Steve
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Long Lines techniques
Just for you Bob, I will make this short.
Here is the most bizarre solution as you might agree.
Edit a file of python code. Use lines as long as it takes. Make sure it works.
Edit the file again and split lines in a way that does not mess up indentation.
Meaning, use a normal text editor, not one that understands python.
At the end of each line you cut, place a marker that makes sense. For example
“!EOL” that may be easier to read or print (or harder) but not to edit.
Save all this to another plain file. It is not valid python.
Write a real python file that contains a function with a name like
outclude(filename) that opens a file, reads in the entire file, replaces every
place with the marker at the end of the line with NOTHING, effectively splicing
it to the next line. It then returns a string containing your entire program
with proper indentation.
Finally:
eval(outclude("filename"))
SKIPPING the other 98% this post could have had. Just an example:
DISCLAIMER: The above was posted as a sick form of humor, only. Any resemblance
to reality is unintentional. But note, some languages do have a phase of
scanning that would allow tricks like this in a pre-processor.
From: Bob Gailer
Sent: Thursday, December 13, 2018 2:33 PM
To: Avi Gross
Cc: tutor@python.org
Subject: Re: [Tutor] Long Lines techniques
On Dec 13, 2018 1:51 PM, "Avi Gross" mailto:avigr...@verizon.net> > wrote:
>
> Simple question:
Avi: when I see an email from you I tend to ignore it because it always seems
to lead to something that is long, time consuming and complex. Would you
consider finding ways to make your questions or comments a lot briefer?
I will be more inclined to read them if they are briefer.
You have correctly determined the conditions that will lead to continuation
lines without backslash. I think we tend to use whatever is convenient.
In your example of a list comprehension over multiple lines there is no
indentation. There is just a lot of white space. You might look at it this way:
the compiler sees a left bracket with no corresponding right bracket on that
line. So it assumes that there's more to the statement on the next line, it
ignores the newline and just continues. Indentation is only significant if it
starts at the beginning of a statement.
Hope this helps
Bob gailer
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Long Lines techniques
On 13/12/2018 17:36, Avi Gross wrote:
> When lines get long, what points does splitting them make sense and what
> methods are preferred?
Its down to personal preference and convenience
plus a smidge of idiom.
> Yes, I am aware of ways to break up something long by breaking in into
> multiple statements. It feels silly to do this:
>
> A = "long string"
> B = "another string"
> .
> Z = "last string"
I'd probably suggest
stgs = ''.join([
"long string",
"another string",
...
"last string"
])
> There are places you can break lines as in a comprehension such as this set
> comprehension:
> letter_set = { letter
>for word in (left_list + right_list)
>for letter in word }
You can break anythiong anywhere (pretty much) inside a set of
parens/brackets/braces.
> So clearly when you use [.] it ignores indentation until it gets to the end.
Indentation only matters for blocks, inside parens etc it
doesn't signify.
> But python has a few features like allowing a comma at the end of a tuple
> that get in the way:
>
x,y = 1,
There's no parens to hold the subsequent line.
You have expressed a valid single value tuple.
> One that seems not to be favored is a backslash at the end of a line:
x = 1 + \
> 2 + \
> 3
Its easy to get wrong and rarely needed but if its the
only way to get neat code its acceptable.
> I will stop here with saying that unlike many languages, parentheses must be
> used with care in python as they may create a tuple
Not a tuple, they arecreated with commas.
But parens are used to disambiguate tuples
- for example in argument lists. But its the comma that makes it a tiuple:
t = 5,6 # 5,6 is a tuple
a,b = t # a->5, b->6
> or even generator expression.
These are usually just parens in a function. The generator
expression is the bit inside (not including) the parens.
> Curly braces may make a set or dictionary. Options for splitting
> the code without messing up indenting cues would be nice to understand.
Braces and square brackets do create structures,
but you can break lines within those. Indeed its very
common to define large literal structures across
multiple lines (possibly including descriptive comments):
mydict = {
"key" : 42, # the first key
"two" : 66 # another key
...
}
HTH
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Extract URL
On 13/12/2018 17:14, Sammy Lee wrote: > I need help on the problem stated below. > > > Given a URL, open the webpage and return the first anchor link url (a href). You will find a very similar example in the Web Client topic of my tutorial, see below... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python function
On 13/12/2018 17:21, Sammy Lee wrote: > How do I create a python function that opens a CSV file and determines how > many columns > of data are in the file? The CSV files have been randomly generated from > https://www.mockaroo.com/ > > def csv_column_count(openfile): You will find a bunch of stuff on creating functions in the Functions and Modules topic of my tutorial.(see below) You will find a bunch of stuff on handling files in the Handling Files topic of my tutorial.(see below) You will find a lot of help on using CSV files in the csv module documentation in the standard library. (Or you could buy my book "Python Projects" which has a section on the topic.) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Import CSV
On 13/12/2018 16:56, Sammy Lee wrote: > I need help on how to open a webpage and save the HTML to a given file path, > given a URL. You can use the standard library urllib module to fetch the file but most folks find the requests module easier to use. requests is a 3rd party module you will need to install. You can read about how to use urllib in the Web Clients topic of my tutorial(see below). Saving the HTML is just the same as saving any other string to a file. Do you know how to do that? If not the file handling topic of my tutor covers it. Your subject mentions CSV but your text only says HTML. If you do have a CSV file then the Python csv module will deal with parsing the CSV data for you. The module documentation is quite helpful. If you need to pare HTML to extract the data then the standard http module can help there. See the Web Clients topic in my tutorial for more on that too. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Long Lines techniques
On Thu, Dec 13, 2018 at 12:36:27PM -0500, Avi Gross wrote:
> Simple question:
>
> When lines get long, what points does splitting them make sense and what
> methods are preferred?
Good question!
First, some background:
Long lines are a potential code smell: a possible sign of excessively
terse code. A long line may be a sign that you're doing too much in one
line.
https://martinfowler.com/bliki/CodeSmell.html
http://wiki.c2.com/?CodeSmell
https://blog.codinghorror.com/code-smells/
Related:
https://www.joelonsoftware.com/2005/05/11/making-wrong-code-look-wrong/
Note that merely splitting a logical line over two or more physical
lines may still be a code-smell. Sure, your eyes don't get as tired
reading fifteen lines of 50 characters each, compared to a single 750
character line, but there's just as much processing going on in what is
essentially a single operation.
Long lines are harder to read: your eyes have to scan across a long
line, and beyond 60 or 70 characters, it becomes physically more
difficult to scan across the line, and the error rate increases.
[Citation required.]
But short lines don't include enough information, so the traditional
compromise is 80 characters, the character width of the old-school
green-screen terminals. The Python standard library uses 79 characters.
(The odd number is to allow for scripts which count the newline at the
end of the line as one of the 80.)
https://www.python.org/dev/peps/pep-0008/
Okay, so we have a style-guide that sets a maximum line length, whether
it is 72 or 79 or 90 or 100 characters. What do you do when a line
exceeds that length?
The only firm rule is that you must treat each case on its own merits.
There is no one right or wrong answer. Every long line of code is
different, and the solution will depend on the line itself. There is no
getting away from human judgement.
(1) Long names. Do you really need to call the variable
"number_of_characters" when "numchars" or even "n" would do?
The same applies to long function names: "get_data_from_database" is
probably redundant, "get_data" will probably do.
Especially watch out for long dotted names that you use over and over
again. Unlike static languages like Java, each dot represents a runtime
lookup. Long names like:
package.subpackage.module.object.method
requires four lookups. Look for oportunities to make an alias for a long
name and avoid long chains of dots:
for item in sequence:
do_something_with(package.subpackage.module.object.method(arg, item))
can be refactored to:
method = package.subpackage.module.object.method
for item in sequence:
do_something_with(method(arg, item))
and is both easier to read and more efficient. A double win!
(2) Temporary constants: sometimes it is good enough to just introduce a
simple named constant used once. The cognitive load is low if it is
defined immediately before it is used. Instead of the long line:
raise ValueError("expected a list, string, dict or None, but instead got
'%s'" % type(value).__name__)
I write:
errmsg = "expected a list, string, dict or None, but instead got '%s'"
raise ValueError(errmsg % type(value).__name__)
(3) Code refactoring. Maybe that long line is sign that you need to add
a method or function? Especially if you are using that line, or similar,
in multiple places. But refactoring is justified even if you use the
line *once* if it is complicated enough.
Likewise, sometimes it is helpful to factor out separate sub-expressions
onto their own lines, using their own variables, rather than doing
everything in a single, complicated, expression.
Psychologists, educators and linguists call this "chunking", and it is
often very helpful for simplifying complicated ideas, sentences and
expressions.
The lack of chunks is why long Perl one-liners are so inpenetrable.
(4) Split the long logical line over multiple physical lines. This does
nothing to reduce the inherent complexity of the line, but if that's
fairly low to start with, it is often helpful.
Python gives us two ways to split a logical line over multiple physical
lines: a backslash at the end of the line, and brackets of any sort.
The preferred way is to use round brackets for grouping:
result = (some very long expression
which can be split over
many lines)
This is especially useful with function calls:
result = function(first_argument, second_argument,
third_argument, fourth_argument)
If you are building a list or dict literal, there is no need for the
parentheses, as square and curly brackets have the same effect. That's
especially useful with two-dimensional nested lists:
data = [[row, one, with, many, items],
[row, two, with, many, items],
[row, three, with, many, items]]
For long strings, I like to use *implicit string concatentation*. String
literals which are separated by
Re: [Tutor] Import CSV
Take a look at the book Automating the Boring Stuff with Python (free PDF): https://automatetheboringstuff.com/ There's a couple of chapters in there for downloading CSV files and extracting the data. On Thu, Dec 13, 2018 at 1:57 PM Sammy Lee wrote: > I need help on how to open a webpage and save the HTML to a given file > path, given a URL. > ___ > Tutor maillist - Tutor@python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Long Lines techniques
On 13/12/2018 19:33, Bob Gailer wrote: On Dec 13, 2018 1:51 PM, "Avi Gross" wrote: Simple question: Avi: when I see an email from you I tend to ignore it because it always seems to lead to something that is long, time consuming and complex. Would you consider finding ways to make your questions or comments a lot briefer? I will be more inclined to read them if they are briefer. You have correctly determined the conditions that will lead to continuation lines without backslash. I think we tend to use whatever is convenient. In your example of a list comprehension over multiple lines there is no indentation. There is just a lot of white space. You might look at it this way: the compiler sees a left bracket with no corresponding right bracket on that line. So it assumes that there's more to the statement on the next line, it ignores the newline and just continues. Indentation is only significant if it starts at the beginning of a statement. Hope this helps Bob gailer Big +1 from me :-) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Open webpage and save CSV
On 13/12/2018 17:16, Sammy Lee wrote: I need help on the problem below, Given a URL, open the webpage and save the CSV to a given file path. def save_url_to_csv_file(url, savefile): Run up your favourite text editor and type up some code. Try to run it. Tell us what problems you get. Failing that send a large cheque to the PSF and no doubt somebody will help you out. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Extract URL
On Dec 13, 2018 2:01 PM, "Sammy Lee" wrote: > > I need help on the problem stated below. > > > Given a URL, open the webpage and return the first anchor link url (a href). > > > def extract_url_link(url): Same comments as my other two emails. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Python function
On Dec 13, 2018 1:55 PM, "Sammy Lee" wrote: > > How do I create a python function that opens a CSV file and determines how many columns > of data are in the file? The CSV files have been randomly generated from https://www.mockaroo.com/ > > def csv_column_count(openfile): Same comments as I made in response to your other question. What part of this do you need help with? do need to know how to read a file? do you need to understand what a CSV file is? do you need to know how to parse a character string? The more specific you are the easier it is for us to help you. Have you ever written a Python program? What has your course taught you so far? Do you know how to write pseudocode? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Long Lines techniques
On Dec 13, 2018 1:51 PM, "Avi Gross" wrote: > > Simple question: Avi: when I see an email from you I tend to ignore it because it always seems to lead to something that is long, time consuming and complex. Would you consider finding ways to make your questions or comments a lot briefer? I will be more inclined to read them if they are briefer. You have correctly determined the conditions that will lead to continuation lines without backslash. I think we tend to use whatever is convenient. In your example of a list comprehension over multiple lines there is no indentation. There is just a lot of white space. You might look at it this way: the compiler sees a left bracket with no corresponding right bracket on that line. So it assumes that there's more to the statement on the next line, it ignores the newline and just continues. Indentation is only significant if it starts at the beginning of a statement. Hope this helps Bob gailer ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Open webpage and save CSV
On Dec 13, 2018 1:51 PM, "Sammy Lee" wrote: > > I need help on the problem below, Could you be more specific? What kind of help do you need? Have you made any attempt to write a program? We are happy to help but we're not going to do all the work for you. So tell us what you do know about the various aspects of this problem. My personal guess is that this is a homework assignment. If that's the case the class should have given you some of the information (ideally all the information) you need to solve the problem. Also please tell us what operating system you're using and what version of python. You will need the services of the urllib. request module to get the contents of a web page. So start with that. > Given a URL, open the webpage and save the CSV to a given file path. > > > def save_url_to_csv_file(url, savefile): Bob Gailer ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Extract URL
I need help on the problem stated below. Given a URL, open the webpage and return the first anchor link url (a href). def extract_url_link(url): ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Import CSV
def save_url_to_file(url, savefile): This is the starting code. > On December 13, 2018 at 11:56 AM Sammy Lee wrote: > > > I need help on how to open a webpage and save the HTML to a given file > path, given a URL. > ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Python function
How do I create a python function that opens a CSV file and determines how many columns of data are in the file? The CSV files have been randomly generated from https://www.mockaroo.com/ def csv_column_count(openfile): ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Import CSV
I need help on how to open a webpage and save the HTML to a given file path, given a URL. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Open webpage and save CSV
I need help on the problem below, Given a URL, open the webpage and save the CSV to a given file path. def save_url_to_csv_file(url, savefile): ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Long Lines techniques
Simple question:
When lines get long, what points does splitting them make sense and what
methods are preferred?
Details.
I am used to many languages where you can continue a statement across
multiple lines. What they share in common is the fact they do not use
indenting for the use Python makes of it. They often use something like
"{.}" to combine multiple statements into a single body and carriage returns
are generally ignored as multiple lines are parsed to search for thee end of
the compound statement at the "}" . In many languages, parentheses and
square brackets and even angle brackets can serve a similar function. Some
languages that insist on ending a statement with ";" are even looser. Many
treat a trailing comma or plus sign (and other symbols) as a hint to
continue reading another line.
So I would like some feedback on ways to deal with longer statements in
Python.
Yes, I am aware of ways to break up something long by breaking in into
multiple statements. It feels silly to do this:
A = "long string"
B = "another string"
.
Z = "last string"
When you meant to write one long string in the first place or connect a
bunch of them with adjacency or a "+"
By breaking them into snippets like above, and using short variable names,
you can combine them like this:
Combined = A + B + . + Z
Or even worse:
Combined = ""
Combined += A
Combined += B
.
Combined += Z
The above was ONE example of many where I bend my code in ways to make the
lines reasonably short so reading and editing are easy.
There are places you can break lines as in a comprehension such as this set
comprehension:
letter_set = { letter
for word in (left_list + right_list)
for letter in word }
The above is an example where I know I can break because the {} is holding
it together. I know I can break at each "for" or "if" but can I break at
random places? Here is a bizarre example that worked:
>>> [ x *
y
for x in
range(1,5,2)
for y
in range(2,
6
,2)
if x + y <
9
]
[2, 4, 6, 12]
So clearly when you use [.] it ignores indentation until it gets to the end.
I see that parentheses sometimes offer a similar protection as in the new
print function:
>>> print(1,
2
,
3)
1 2 3
But python has a few features like allowing a comma at the end of a tuple
that get in the way:
>>> x,y = 1,
Traceback (most recent call last):
File "", line 1, in
x,y = 1,
ValueError: not enough values to unpack (expected 2, got 1)
DUH! I was going to put the rest on the next line!
So I added parentheses and that does the trick here.
>>> x,y = (1,
2
)
Similarly, it is a syntax error if I simply end with something like a +
>>> x = 1 +
SyntaxError: invalid syntax
>>> x = (1 +
2)
>>> x
3
I AM NOT SAYING THIS IS HORRIBLE, just asking what pythonic ways are best to
use to get around things.
One that seems not to be favored is a backslash at the end of a line:
>>> x = 1 + \
2 + \
3
>>> x
6
Although my examples use short numbers, remember I might be using this with
long lines and not want to mess up the overall indentation. Strings have a
similar issue:
>>> x = "hello
SyntaxError: EOL while scanning string literal
>>> x = """hello
world"""
>>> x
'hello\nworld'
>>> x = "hello \
world!"
>>> x
'hello \tworld!''
As shown, the triple quotes sort of work but introduce and keep carriage
returns and formatting inside. The backslash at the end suppressed the
former but keeps the latter. Yes, there are routines you can pass the string
through afterward to remove leading whitespace or carriage returns.
I will stop here with saying that unlike many languages, parentheses must be
used with care in python as they may create a tuple or even generator
expression. Curly braces may make a set or dictionary. Options for splitting
the code without messing up indenting cues would be nice to understand.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
