Re: [Tutor] I want to learn how memory works!

[Alan Gauld via Tutor]

On 29/09/17 19:47, Michael C wrote:
> Could you point me to a source of information about all the things someone
> should know before he starts writing a memory scanner?  

Not a single source but wikipedia is a good start for anything technical.

In particular you need to understand the difference between
virtual(VM) and physical memory(PM). Virtual memory is the memory
that your program(process in OS speak) sees and physical memory
is the RAM installed in your computer. The OS maps virtual memory
to physical memory and depending on the OS that mapping can be
done in many ways.

In some OS (especially mainframes) you can specify in a config
file how much VM each process is given at startup, in others you
specify how much VM it needs so the OS won't allow it to start
up unless there is that much available (this is often used on
small machines and embedded systems). In others the VM is always
a theoretical space determined by the address size (or built
into the kernel).

The PM is a combination of the theoretical address space, the
installed RAM and the virtual memory page file(s). The OS swaps
memory between RAM and page file as necessary. It is quite a
complex topic and heavily OS dependent.

It looks like you are using Windows and it's too long since
I looked at that level of detail (around NT4!) to be confident
of a reply, but ~ I'd start with wikipedia for the basic
concepts then move to MSDN for the detail for your OS.

The key point is that from inside a process you are seeing a
virtualized version of memory, rarely, if ever, the actual
physical RAM addresses. Only the OS sees that.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] OT: how to best follow the posting style on this list

[boB Stepp]

On Fri, Sep 29, 2017 at 9:55 AM, Albert-Jan Roskam
 wrote:
>
> Hi,
>
> I often read messages to this list from my Android from the mobile Hotmail 
> page. On other occasions I would read from my browser, from a Windows or a 
> Linux desktop. Howver, Hotmail pretty much sucks when it comes to obeying the 
> posting style [1]. It's even worse than Yahoo mail, which I've also tried 
> (until they got hacked). Is there a recommended way to do this?  Maybe gmail? 
> I prefer to use more or less the same interface across platforms and devices, 
> but this is not crucial. I've also been looking for pan-like apps (not 
> necessarily free) for my Phone, but I couldn't find one that worked. Perhaps 
> because I used a free news server (my ISP doesn't have one).

This is something I have struggled with as well.  For reading purposes
it is usually not much of a problem.  It is when replying to posts
that the trouble begins.  I have settled into Gmail over time, mostly
because it seems to work with my Android phone, iPad, desktop (Linux
or Windows), etc., where my primary goal is to have all of my contact
information well and easily synced.  But I have been unable to to
coerce it to do plain text responses on my iPad for instance.  I used
to be able to do this on my phone, but I have been through at least
two major Android updates to my phone and now the way is not obvious,
if it exists, anymore (Not that I wish to reply to Tutor posts much
from my phone!).  However, I have seen other members of this list
occasionally responding from their phones who are Gmail users, so
perhaps I am being dense.  And even with my desktop usage of Gmail
(Using the Chrome browser in Windows and Firefox for Linux),
occasionally updates have broken my settings and I have had to fiddle
around to regain a fixed width font and default plain text email
settings.

So if there is a better way that would make things uniformly *easy*
across all types of common devices, I would be interested, too!  I
wonder what the people at Google use for lists like Tutor?  I believe
there are some such people on the main list.

-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Python programming for the absolute beginner

[boB Stepp]

On Fri, Sep 29, 2017 at 1:20 PM, Alan Gauld via Tutor  wrote:
> On 29/09/17 08:51, Peter Collidge wrote:
>> I have borrowed the above book from my local library but I believe it was
>> written in 2010 and as a result I am having difficulty in deciding which
>> version of Python to download.
>> Can anyone help?
>
> If you want to follow the book use the version the book
> uses - probably 2.6 or something close?

I no longer have a copy of this book, but I am fairly certain it uses
an early version of Python 3.  The main problem, though, is it uses a
customized version of Pygame for its later gaming code called
Livewires, so that would have to be Python-version-compatible.  IIRC,
the author had a download site where you could get the whole package
together:  correct Python version used in book, correct Livewires
version, etc., and that this was addressed in the book (Maybe an
appendix?  Or the first get things setup chapter?  Can't remember for
sure.)

If the OP goes on to use the sequel to this book, named "More Python
Programming for the Absolute Beginner" (By a different author.),
*that* book uses straight-up Pygame, again with an early version of
Python 3.  Again, I don't think there will be any problems with the
exact version of Python 3 used as long as the version of Pygame is
compatible with the chosen Python.



-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] OT: how to best follow the posting style on this list

[Mats Wichmann]

On 09/30/2017 08:37 AM, boB Stepp wrote:
> On Fri, Sep 29, 2017 at 9:55 AM, Albert-Jan Roskam
>  wrote:
>>
>> Hi,
>>
>> I often read messages to this list from my Android from the mobile Hotmail 
>> page. On other occasions I would read from my browser, from a Windows or a 
>> Linux desktop. Howver, Hotmail pretty much sucks when it comes to obeying 
>> the posting style [1]. It's even worse than Yahoo mail, which I've also 
>> tried (until they got hacked). Is there a recommended way to do this?  Maybe 
>> gmail? I prefer to use more or less the same interface across platforms and 
>> devices, but this is not crucial. I've also been looking for pan-like apps 
>> (not necessarily free) for my Phone, but I couldn't find one that worked. 
>> Perhaps because I used a free news server (my ISP doesn't have one).

> 
> So if there is a better way that would make things uniformly *easy*
> across all types of common devices, I would be interested, too!  I
> wonder what the people at Google use for lists like Tutor?  I believe
> there are some such people on the main list.
> 

Thanks for asking, by the way!

Honestly, those of us who care about the stuff in the reference are more
and more being considered dinosaurs... email has pretty much moved on
and almost finished leaving behind the plain-text option.  When I worked
for Samsung for a while, most of the company had to use an in-house
developed mail system as it was the only "approved" one, and it simply
had no option for not emitting styled emails (my organization as
specifically charged with working in the open source world got an
exemption, but that was highly rare).  No I don't know how other
companies deal with the "participate in external mailing list" scenario,
there's certainly enough of them.

I don't have good answers.  I use K-9 on my android phone and it does
fine, although occasionally throws up on _received_ emails with excess
crud. You can set it up as a client of your gmail account if you wish,
you don't have to use gmail to read gmail!  Actually have more trouble
on the desktop: I'm on some mailing list where almost all of the
participants work in Outlook-only shops, and we'll get things like "my
responses interleaved in blue" (guess what: Thunderbird on my Linux
desktop set up the way I have it doesn't show those).  Sometimes get
some funky apparently Outlook-only list style markup come through as
plain text rendering things completely unreadable.  Big sigh...
absolutely nothing I can do about the mess, just have to grit my teeth
and muddle through.

My surviving advice is "do the best you can", wish it was better. You
can look at a message or two you've sent in an archive and see how bad
it comes out, unfortunately reading your own email in the client that
sent it doesn't usually accurately reflect what it will look like to others.
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Directory Structure

[Chris]

Thank you for your detailed reply! I've tried to explain in my mails to
Alan and Mats what I'm trying to achieve.

On Sat, 30 Sep 2017 11:32:57 +1000
Steven D'Aprano  wrote:

> On Fri, Sep 29, 2017 at 07:02:07PM +0200, Chris wrote:
> 
> > Background: Maildirs with mails older than five years should be
> > archived. The folder structure should be kept in the target.  
> 
> Archived to what?

A separate Maildir on tape.

> > I was very surprised, that there seems no readily usable module
> > available. (In Perl neither).  
> 
> Reusable module to do what *precisely*? If you cannot explain what
> you need, how do you expect somebody to have anticipated your
> requirements and written a module to do it?

Represent the structure in memory.
 
> > What's the best way to save them?  
> 
> Depends on what you are doing. But coding the paths in your source
> code is almost certainly not what you want to do. Surely you want to
> read the paths from the maildir itself, as it *actually* exists,
> rather than try to hard-code what you expect it to be in your source
> code?

Well, if I had the structure in memory, I could save additional
information and could print different lists, e.g. sorted by attachment
size, sorted by project. A project can appear in different places
in the tree.

> Have you looked at the contents of a maildir? Its actually an almost 
> flat structure. Nested mail folders are not nested on the disk: a 
> user's mail folder structure that looks like:
> 
> inbox
> sent
> trash
> personal
> ├── family
> └── friends
> work
> ├── critical
> ├── important
> └── low
> 
> 
> is stored on disk as:
> 
> Maildir/
> ├──  .sent/
> ├──  .trash/
> ├──  .personal/
> ├──  .personal.family/
> ├──  .person.friends/
> ├──  .work
> ├──  .work.critical
> ├──  .work.important
> └──  .work.low

Good objection. You can make dovecot use the first layout on disk.
Probably not a gain in disguise.
 
> So all you really need is to record the path to the top level maildir 
> directories (the original, and the place where you are archiving
> them). The subdirectories, you read from the disk as you go.

Ok, I could even do this with the first structure. 
 
> Actually, *you* don't read them at all. Have you looked at the
> mailbox module in the standard library? It supports Maildir. I expect
> that what you would do is something like:
> 
> source = Maildir('path/to/source')
> archive = Maildir('path/to/archive')
> for each directory in source:
> for each mail in directory:
> if mail older than five years:
> copy mail to archive
> delete mail from source

I've used os.walk. I'll have a look at the Maildir library.


- Chris
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Most common words in a text file

[Cameron Simpson]

On 30Sep2017 22:42, Sri G.  wrote:

I’ve written the code below for finding the most common words in a text
file that has about 1.1 million words. It's working fine, but I believe
there is always room for improvement.

When run, the function in the script gets a text file from the command-line
argument sys.argv[1], opens the file in read mode, converts the text to
lowercase, makes a list of words from the text after removing any
whitespaces or empty strings, and stores the list elements as dictionary
keys and values in a collections.Counter object. Finally, it returns a
dictionary of the most common words and their counts. The
words.most_common() method gets its argument from the optional top
parameter.


Thank you, this is a nice clear explaination.


import collections


It is common to import specific names from modules; whether you just import the 
module or specific names depends on your needs. However, in this case, instead 
of importing "collections" and creating a counter with:


 collections.Counter()

I would import the name "Counter" and use it like this:

 from collections import Counter
 ...
 ... Counter() ...


def find_most_common_words(textfile, top=10):


It is often good form to externalise the "10" as a constant (Python doesn't 
have constants per se, but it has a convention for value which would be 
constants in other languages):


 DEFAULT_TOP_SIZE = 10

 def find_most_common_words(textfile, top=DEFAULT_TOP_SIZE):

In a library of functions you can then put all the "constants" up the top where 
they can be seen or modified, and reference then lower down where the functions 
are defined.



   ''' Returns the most common words in the textfile.'''
   textfile = open(textfile)


You shouldn't reuse "textfile" as a file. Keep the filename value distinct. I 
tend to use names like "textpath" for the filename and "textfile" for the open 
file, eg:


 textfile = open(textpath)

Of course you change the parameter name to match.


   textfile = open(textfile)
   text = textfile.read().lower()
   textfile.close()


The preferred way of opening files looks like this:

 with open(textpath) as textfile:
   ... work on the data from textfile ...
   ...
   ...

This automatically closes textfile on exiting the "with" suite, even if an 
exception occurs (which is very handy in situations where the exception might 
be caught in outer code - you never miss closing the file). It is also shorter 
and more readable.



   text = textfile.read().lower()


This code reads the entire text into memory. It is more frugal to read the file 
progressively, which is good because it scales to files of any size. This 
concern is prominent in my mind because I started programming on systems with a 
16 bit address space, so 64K max including the program itself _and_ the OS. But 
it applies even today when many data files are very large.


So one might process file file as lines of text:

 C = Counter()
 for line in textfile:
   words = line.lower().split()
   C.update(words)

This avoids the need for more than a single line to be stored in memory at any 
given time.


Cheers,
Cameron Simpson  (formerly c...@zip.com.au)
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Most common words in a text file

[Sri G.]

I'm learning programming with Python.

I’ve written the code below for finding the most common words in a text
file that has about 1.1 million words. It's working fine, but I believe
there is always room for improvement.

When run, the function in the script gets a text file from the command-line
argument sys.argv[1], opens the file in read mode, converts the text to
lowercase, makes a list of words from the text after removing any
whitespaces or empty strings, and stores the list elements as dictionary
keys and values in a collections.Counter object. Finally, it returns a
dictionary of the most common words and their counts. The
words.most_common() method gets its argument from the optional top
 parameter.

import sysimport collections
def find_most_common_words(textfile, top=10):
''' Returns the most common words in the textfile.'''

textfile = open(textfile)
text = textfile.read().lower()
textfile.close()
words = collections.Counter(text.split()) # how often each word appears

return dict(words.most_common(top))

filename = sys.argv[1]
top_five_words = find_most_common_words(filename, 5)

I need your comments please.

Sri
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Am I missing something obvious about "FizzBuzz"?

[boB Stepp]

On the main Python list there is a thread entitled, "Beginners and
experts (Batchelder blog post)", and Stefan Ram mentioned:  "... The
fact that programmers can't program is known since the invention of
the "FizzBuzz" programmer test. ..."

I vaguely recall seeing some reference to "FizzBuzz" in my past
Internet wanderings, but did not recall any specifics, so did a search
for it.  My first hit was http://wiki.c2.com/?FizzBuzzTest  So I went
there and was informed:


The "Fizz-Buzz test" is an interview question designed to help filter
out the 99.5% of programming job candidates who can't seem to program
their way out of a wet paper bag. The text of the programming
assignment is as follows:"Write a program that prints the numbers from
1 to 100. But for multiples of three print “Fizz” instead of the
number and for the multiples of five print “Buzz”. For numbers which
are multiples of both three and five print “FizzBuzz”."


I did *not* read any further, opened up gVim, and in less than 5 minutes had:

for i in range(1, 101):
if i % 15 == 0:
print('FizzBuzz')
elif i % 3 == 0:
print('Fizz')
elif i % 5 == 0:
print('Buzz')
else:
print(i)

If I understand the problem correctly, this gives the desired result.
I would have had this in < 30 seconds if I did not stop at each stage
and verified I was getting my expected result.  I don't claim to be a
*real* programmer and am definitely no where approaching a
professional programmer, but this question seems ridiculously easy!
Surely I am missing something obvious and substantial?  Of course, I
could have done tests, comments, prettied up the output, etc., but the
essential problem seems to be trivial for *anyone* exposed to the if -
elif - else structure and is aware of the modulus operator.  What am I
missing???  How can anyone in any programming language not get this
question right?

-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] logging to cmd.exe

[boB Stepp]

On Tue, Sep 26, 2017 at 6:22 AM, Albert-Jan Roskam
 wrote:


> PS: sorry about the missing quote (>>) markers. Hotmail can't do this. Is 
> Gmail better?

Yeah, in Gmail it will handle the quote markers when doing plain text.


-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Am I missing something obvious about "FizzBuzz"?

[Steven D'Aprano]

On Sun, Oct 01, 2017 at 12:09:55AM -0500, boB Stepp wrote:

> If I understand the problem correctly, this gives the desired result.
> I would have had this in < 30 seconds if I did not stop at each stage
> and verified I was getting my expected result.  I don't claim to be a
> *real* programmer and am definitely no where approaching a
> professional programmer, but this question seems ridiculously easy!

Congratulations, according to some, you are a better programmer than 99% 
of self-described programmers.

https://blog.codinghorror.com/why-cant-programmers-program/

For a dissenting view:

https://www.skorks.com/2010/10/99-out-of-100-programmers-cant-program-i-call-bullshit/


-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Am I missing something obvious about "FizzBuzz"?

[boB Stepp]

On Sun, Oct 1, 2017 at 12:32 AM, Steven D'Aprano  wrote:
> On Sun, Oct 01, 2017 at 12:09:55AM -0500, boB Stepp wrote:
>
>> If I understand the problem correctly, this gives the desired result.
>> I would have had this in < 30 seconds if I did not stop at each stage
>> and verified I was getting my expected result.  I don't claim to be a
>> *real* programmer and am definitely no where approaching a
>> professional programmer, but this question seems ridiculously easy!
>
> Congratulations, according to some, you are a better programmer than 99%
> of self-described programmers.

I definitely was *not* looking for a pat on the back.  I just could
not believe that "FizzBuzz" (Or similar questions.) would ever be
needed in a job interview for programming/software engineering.  I
honestly thought that I was missing something.  I know that I have had
some really bad *blooper* moments in the past on this list!

> https://blog.codinghorror.com/why-cant-programmers-program/

I truly hope that the above article does not reflect reality!

> For a dissenting view:
>
> https://www.skorks.com/2010/10/99-out-of-100-programmers-cant-program-i-call-bullshit/

Hopefully this one provides a more accurate picture, though it is not
a rosy view either.

The more I read and study, the more I follow this and the main list,
the more I realize how truly little I know ...

-- 
boB
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor