Re: [Tutor] (no subject)

[Steven D'Aprano]

On Sun, Nov 11, 2018 at 03:00:00PM -0800, Stealth Fleet wrote:

[...]
> tokenAmount works but the buy and cashIn are not being ran why?

You define the function, but never call it. Functions aren't called 
automatically.

py> def example():
... print("calling example")
...
py>
py> # nothing is happening yet...
...
py> example()  # call the function
calling example


> When I put
> "print(buy, cashIn)" it gives me a long message that ends in an error

Would you like to tell us that message, or would you prefer that we make 
a wild guess?

Please COPY AND PASTE (don't summarise or retype from memory) the ENTIRE 
error message, not just the last line, and especially DON'T take a 
screen shot or photo and send that.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] saveLine decked

[Avi Gross]

Peter,

Appreciated. I wrote something like this in another message before reading
yours. Indeed one of the things I found was the deque class in the
collections module. 

But I was not immediately clear on whether that would be directly
applicable. Their maximum sounded like if you exceeded it, it might either
reject the addition or throw an error. The behavior I wanted was sort of a
sliding window protocol where the oldest entry scrolled off the screen or
was simply removed. Sort of like what you might do with a moving average
that takes the average of just the last 20 days of a stock price.

But I searched some more and stand corrected. 

"New in version 2.4.

If maxlen is not specified or is None, deques may grow to an arbitrary
length. Otherwise, the deque is bounded to the specified maximum length.
Once a bounded length deque is full, when new items are added, a
corresponding number of items are discarded from the opposite end. Bounded
length deques provide functionality similar to the tail filter in Unix. They
are also useful for tracking transactions and other pools of data where only
the most recent activity is of interest."

That sounds exactly like what is needed. As long as you keep adding at the
end (using the append method) it should eventually remove from the beginning
automatically.  No need to use count and selectively remove or pop manually.

And despite all the additional functionality, I suspect it is tuned and
perhaps has parts written in C++ for added speed. I do note that any larger
log file used in the application discussed may throw things on the deque
many times but only ask it to display rarely so the former should be
optimized.

But one question before I go, Columbo style. The manual page
(https://docs.python.org/2/library/collections.html ) suggest you call deque
with an iterator. That would not necessarily meet our need as giving it the
entire file as an iterator would just grind away without any logic and keep
just the last N lines. We could arrange the logic in our own iterator, such
as a function that reads a line at a time using its own open iterator and
yields the line over but that too is problematic as to how and when you stop
and print the results. But on second look, the iterator is optional and I
tried creating a deque using just a maxlen=3 argument for illustration.

>>> from collections import deque

>>> a=deque(maxlen=3)
>>> a
deque([], maxlen=3)
>>> a.append('line 1\n')
>>> a
deque(['line 1\n'], maxlen=3)
>>> a.append('line 2\n')
>>> a.append('line 3\n')
>>> a
deque(['line 1\n', 'line 2\n', 'line 3\n'], maxlen=3)
>>> a.append('line N\n')
>>> a
deque(['line 2\n', 'line 3\n', 'line N\n'], maxlen=3)

OK, that looks right so all you need to figure out is how to print it in a
format you want.

As it happens, deque has an str and a repr that seem the same when I try to
print:

>>> a.__str__()
"deque(['line 2\\n', 'line 3\\n', 'line N\\n'], maxlen=3)"
>>> a.__repr__()
"deque(['line 2\\n', 'line 3\\n', 'line N\\n'], maxlen=3)"

So you either need to subclass deque to get your own printable version (or
use an amazing number of other Python tricks since you can, or do something
manually. 

>>> for line in a: print(line)

line 2

line 3

line N

OK, that works but my \n characters at the end of some items might suggest
using end='' in the 3.X version of print for a smaller display.

Summary: the method Peter mentions is a decent solution with no programming
or debugging overhead. It is even flexible enough, if you choose, to store
or display the lines backwards as in showing the last line that showed the
error, followed by successively earlier lines. 

Why use a limited solution when you can play with a full deck?





-Original Message-
From: Tutor  On Behalf Of
Peter Otten
Sent: Sunday, November 11, 2018 2:43 PM
To: tutor@python.org
Subject: Re: [Tutor] saveLine

Avi Gross wrote:

> Alan and others have answered the questions posed and what I am asking 
> now is to look at the function he proposed to keep track of the last 
> five lines.
> 
> There is nothing wrong with it but I wonder what alternatives people 
> would prefer. His code is made for exactly 5 lines to be buffered and 
> is quite efficient. But what if you wanted N lines buffered, perhaps 
> showing a smaller number of lines on some warnings or errors and the 
> full N in other cases?

The standard library features collections.deque. With that:

buffer = collections.deque(maxlen=N)
save_line = buffer.append

This will start with an empty buffer. To preload the buffer:

buffer = collections.deque(itertools.repeat("", N), maxlen=N)

To print the buffer:

print_buffer = sys.stdout.writelines

or, more general:

def print_buffer(items, end=""):
for item in items:
print(item, end=end)

Also, for smallish N:

def print_buffer(items, end=""):
print(*items, sep=end)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription 

[Tutor] Bufferin

[Avi Gross]

After my earlier message (snipped out for space savings) focused on choosing 
among various methods to retain a buffer of recent lines from a file, I 
realized that for many, the best method is simply to import a solution others 
have created, often in the form of an object. Many of the methods I discussed 
(and undoubtedly many other ways) are likely to be incorporated using 
object-oriented aspects in a class. I mean if you created a class called 
Bufferin and set methods like:

__init__ takes an argument "n" with say a default of 5 and initializes whatever 
internal data structure will hold the info.

__add__ might overlay the + sign operator to take the string (or any object) 
passed to it and add it to the internal representation for your ordered 
collection of n objects and perhaps delete something to make room.

__iadd__ and __radd__ might be set to allow += to work as an overlay or to 
accept adding from the other side.

__str__ or perhaps __repr__ could be overlaid to print the results in whatever 
way you want.

And, you could just make normal named methods to allow you to say 
Bufferin.insert() and Bufferin.display() or whatever interface you want to use.

Using an instance of a class this way would let you have multiple active 
buffers active containing the last N lines (or any objects) from multiple 
sources.

Subclassing such an object might even let you be more selective. For example, a 
log may contain entries you do not want buffered such as just warnings or 
entries that say what time something  started. You want the last N lines for 
context to be mostly things that are useful in debugging. Any of the above can 
include specific logic that intercepts the call to add an item and only calls 
the class method if it is deemed to be worth keeping.

In any case, buffering is an extremely common phenomenon, and a search for 
something like "python circular buffer class" can provide code you might just 
include and use for free and that may be well tested or even more efficient. 
Some are probably parts of packages normally available in built-in modules. 

So, again, the solution Alan suggested as a small part of the program he was 
discussing, is fine. I like to get more abstract and see if there is a 
reasonable solution with more power for other needs. My earlier message focused 
on ways to either do it simpler from a human perspective or maybe even faster 
and then on making the size of the buffer variable. Using a class object may 
help not only by having a way to maintain the data without passing it 
repeatedly in functional calls but by allowing multiple buffers running 
independently. Heck, I can even imagine a log where multiple processes or 
threads write to the same log file in an interspersed way. Your goal in the 
program that began this discussion might then be to scan for a problem in ANY 
process or session and as soon as found, display the last N lines for just that 
entity. 

So you might read each line, find something like a process ID, and throw the 
line into an appropriate buffer. One way would be to have a master dictionary 
whose keys are that ID and the values are instances of your buffer object. When 
a failure is encountered, get the object for that key and print the contents as 
discussed above. You might then continue processing the log after optionally 
removing that object or setting it back to empty and process any other such 
errors till the log is completed.

Just some thoughts but also a comment I must add. To those that don't know me 
(meaning almost everyone) I have been known to play games that range from humor 
to sarcasm to word games. I chose the name Bufferin because of the similarity 
in wordplay but also because there is an analgesic product called Bufferin that 
is named that for a somewhat different reason having more to do with Chemistry. 
It is basically Aspirin but with added ingredients that buffer it (Magnesium 
and Calcium carbonates and oxides)  that are antacids so that some people 
experience less gastric discomfort when taking plain aspirin. Alan used "buff" 
as a variable, which has other meanings like someone in good physical shape or 
phrases like being in the buff meaning without clothes. Similar ideas in the 
word "buffed." I generally do not stop and explain my puns or wordplay but if 
something seems phrased oddly, I may be bending things a bit. I did exert some 
self-control by not using the name of my favorite Vampire Slayer 

Last comment. If the purpose of the limited Buffer is to save space by not 
reading the entire file at once and retaining just enough context when needed, 
you may want to remember to release this space too when done using it. Python 
garbage collection may kick in for say closing the file opened in a "for line 
in open(...):" or a context manager context like "with open(...) as line:" but 
you may want to delete things like the buffers manually when done unless they 
too are set up to be temporary. For a 

[Tutor] (no subject)

[Stealth Fleet]

tokenAmount = input( "How many tokens would you like to buy or cash in?:  ")

print (tokenAmount)



def buy ():

if tokenAmount <= 400:

buy = tokenAmount * .2099

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

elif tokenAmount > "400" <= "549":

buy = tokenAmount * .3999

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

elif tokenAmount >= "550" <= "749":

buy = tokenAmount * .4999

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

elif tokenAmount >= "750" <= "999":

buy = tokenAmount * .6299

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")

else:

buy = tokenAmount * .7999

print( " You would like to spend $"+(buy) + "on" + (tokenAmount) +
"tokens.")



def cashIn ():

cashIn = tokenAmount * .05

print( "The amount of money you will receive is $"+ (cashIn))



tokenAmount works but the buy and cashIn are not being ran why? When I put
"print(buy, cashIn)" it gives me a long message that ends in an error any
and all help is greatly appreciated. Sent from Mail
 for Windows 10
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] saveLine

[Peter Otten]

Avi Gross wrote:

> Alan and others have answered the questions posed and what I am asking now
> is to look at the function he proposed to keep track of the last five
> lines.
> 
> There is nothing wrong with it but I wonder what alternatives people would
> prefer. His code is made for exactly 5 lines to be buffered and is quite
> efficient. But what if you wanted N lines buffered, perhaps showing a
> smaller number of lines on some warnings or errors and the full N in other
> cases?

The standard library features collections.deque. With that:

buffer = collections.deque(maxlen=N)
save_line = buffer.append

This will start with an empty buffer. To preload the buffer:

buffer = collections.deque(itertools.repeat("", N), maxlen=N)

To print the buffer:

print_buffer = sys.stdout.writelines

or, more general:

def print_buffer(items, end=""):
for item in items:
print(item, end=end)

Also, for smallish N:

def print_buffer(items, end=""):
print(*items, sep=end)

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] saveLine

[Avi Gross]

Alan and others have answered the questions posed and what I am asking now
is to look at the function he proposed to keep track of the last five lines.

There is nothing wrong with it but I wonder what alternatives people would
prefer. His code is made for exactly 5 lines to be buffered and is quite
efficient. But what if you wanted N lines buffered, perhaps showing a
smaller number of lines on some warnings or errors and the full N in other
cases?

Here is Alan's code for comparison:

buffer = ['','','','','']

def saveLine(line, buff):
buff[0] = buff[1]
buff[1] = buff[2]
buff[2] = buff[3]
buff[3] = buff[4]
buff[4] = line

Again, that works fine. If N was not 5, I would suggest initializing the
buffer might look like this:

buffed = 5
buffer = [''] * buffed

Then instead of changing the list in place, I might change the function to
return the new string created by taking the substring containing all but the
first that is then concatenated with the new entry:

def saveLineN(line, buff):
buff = buff[1:] + [line]
return buff

Clearly less efficient but more general. And, yes, the return statement
could be the entire function as in:

def saveLineN(line, buff):
return  buff[1:] + [line]


Here is a transcript of it running:

>>> buffer
['', '', '', '', '']
>>> saveLineN('a', buffer)
['', '', '', '', 'a']
>>> buffer = saveLineN('a', buffer)
>>> buffer
['', '', '', '', 'a']
>>> buffer = saveLineN('b', buffer)
>>> buffer = saveLineN('c', buffer)
>>> buffer = saveLineN('d', buffer)
>>> buffer = saveLineN('e', buffer)
>>> buffer
['a', 'b', 'c', 'd', 'e']
>>> buffer = saveLineN('6th', buffer)
>>> buffer
['b', 'c', 'd', 'e', '6th']

So perhaps using in-line changes might make sense.

Buff.pop(0) would remove the zeroeth item with the side effect of returning
the first item to be ignored. 

>>> buffer = ['a', 'b', 'c', 'd', 'e']
>>> buffer.pop(0)
'a'
>>> buffer
['b', 'c', 'd', 'e']

And it can be extended in-line:

>>> buffer.append('6th')
>>> buffer
['b', 'c', 'd', 'e', '6th']

Sorry, I mean appended, not extended! LOL!

So to make this compact and less wasteful, I consider using del buffer[0]:

>>> del buffer[0]
>>> buffer
['b', 'c', 'd', 'e']

So here is this version that might be more efficient. It deletes the first
item/line of the buffer then adds a new  nth in-line:

def saveLineN2(line, buff):
del buff[0]
buff.append(line)

Here is a transcript of it in use, using N=3 to be different:

>>> buffed = 3
  
>>> buffer = [''] * buffed
  
>>> buffer
  
['', '', '']
>>> saveLineN2('First Line\n', buffer)
  
>>> buffer
  
['', '', 'First Line\n']
>>> saveLineN2('Second Line\n', buffer)
  
>>> saveLineN2('Third Line\n', buffer)
  
>>> buffer
  
['First Line\n', 'Second Line\n', 'Third Line\n']
>>> saveLineN2('nth Line\n', buffer)
  
>>> buffer
  
['Second Line\n', 'Third Line\n', 'nth Line\n']

I can think of many other ways to do this, arguably some are more weird than
others. There is the obvious one which does all the changes in one line as
in:

buff[0],buff[1],buff[2] = buff[1],buff[2],line

Of course, for 5 you change that a bit. Might even be a tad more efficient.

There is also the odd concept of not scrolling along but dealing with things
at print time. I mean you can have a list with 1:N entries and a variable
that holds an index from 1 to N. You store a new line at buffer[index] each
time then you increment index modulo N. This resets it to 0 periodically. At
print time, you print buffer[index:] + buffer[:index] and you have the same
result.

Does anyone have comments on what methods may be better for some purposes or
additional ways to do this? I mean besides reading all lines into memory and
holding on to them and indexing backwards.

For that matter, you can revisit the question of using a list of lines and
consider a dictionary variant which might work better for larger values of
N. No need to slide a buffer window along, just maintain a modular index and
overwrite the key value as in buffer[index] = line

One more comment, if I may. 

Alan mentions but does not define a printBuffer() function. Using any of the
methods above, you can end up with an error happening early on so the kind
of fixed-length buffer mentioned above contains blank entries (not even a
'\n') so it probably should suppress printing items of zero length or that
are empty. And, in some of the methods shown above, it may be worth starting
with an empty buffer and adding lines up to some N and only then removing
the first entry each time. That would complicate the code a bit but make
printing trivial.





___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Example for read and readlines()

[Alan Gauld via Tutor]

On 11/11/2018 10:04, Asad wrote:

> 1)   and I want to extract the start time , error number and end
> time from this logfile so in this case what should I use I guess option 1  :
> 
> with open(filename, 'r') as f:
> for line in f:
> process(line)

Yes, that woyuld be the best choice in that scenario.

> 2) Another case is a text formatted logfile and I want to print (n-4)
> lines n is the line where error condition was encountered .

In that case you could use readlines() if it is a small file.
Or you could save the last 5 lines and print those each time
you find an error line. You should probably write a function
to save the line since it needs to move the previous lines
up one.

buffer = ['','','','','']

def saveLine(line, buff):
buff[0] = buff[1]
buff[1] = buff[2]
buff[2] = buff[3]
buff[3] = buff[4]
buff[4] = line

for line in file:
saveLine(line,buffer)
if error_condition:
   printBuffer()

readlines is simpler but stores the entire file in memory.
The buffer saves memory but requires some extra processing
to save/print. There are some modules for handling cyclic
stores etc but in a simple case like this they are probably
overkill.

> 3) Do we need to ensure that each line in the logfile ends with \n .
> 
> \n is not visible so can we verify in someway to proof EOL \n is placed
> in the file .

textfile lines are defined by the existence of the \n
so both readlines() and a loop over the file will both
read multiple lines if a \n is missing.

You could use read() and a regex to check for some
text marker and insert the newlines. This would be best
if the whole file had them missing.

If it just an occasional line then you can iterate over
the file as usual and check each line for a missing \n and
insert (or split) as needed.

You might want to write the modified lines back to
a new file.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Example for read and readlines()

[Alan Gauld via Tutor]

On 11/11/2018 09:40, Steven D'Aprano wrote:

>> f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log", 'r' )
> 
> Don't use raw strings r"..." for pathnames.

Umm, Why not?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Example for read and readlines()

[Alan Gauld via Tutor]

On 11/11/2018 06:49, Asad wrote:
> Hi All ,
> 
>  If I am loading a logfile what should I use from the option 1,2,3
> 
> f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log", 'r' )
> 
> 1) should only iterate over f3

This is best for processing line by line which is the most
common way to handle files. It saves memory and allows you
to exit early, without reading the entire file if you are
only looking for say a single entry.

for line in file:
   if terminal_Condition: break
   # process line here

> 2) st = f3.read()

The best solution if you want to process individual characters
or small character groups. Also best if you want to process
the entire file at once, for example using a regular expression
which might span lines.

> 3) st1 = f3.readlines()

Mainly historical and superseded by iterating over the file.
But sometimes useful if you need to do multiple passes over
the lines since it only reads the file once. Very heavy
memory footprint for big files.


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Require Python assistance

[Alan Gauld via Tutor]

On 10/11/2018 18:10, Avi Gross wrote:
> WARNING to any that care:
> 
> As the following letter  is a repeat request without any hint they read the 
> earlier comments here, I did a little searching and see very much the same 
> request on another forum asking how to do this in MATLAB:

The OP has also repeated posted the same message to this list
(which I rejected as moderator).


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] Example for read and readlines()

[Steven D'Aprano]

On Sun, Nov 11, 2018 at 12:19:36PM +0530, Asad wrote:
> Hi All ,
> 
>  If I am loading a logfile what should I use from the option 1,2,3

Depends what you want to do. I assume that the log file is formatted 
into lines of text, so you probably want to iterate over each line.

with open(filename, 'r') as f:
for line in f:
process(line)

is the best idiom to use for line-by-line iteration. It only reads each 
line as needed, not all at once, so it can handle huge files even if the 
file is bigger than the memory you have.


> f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log", 'r' )

Don't use raw strings r"..." for pathnames.


> 1) should only iterate over f3
> 
> 2) st = f3.read()

Use this if you want to iterate over the file character by character, 
after reading the entire file into memory at once.

 
> 3) st1 = f3.readlines()

Use this if you want to read all the lines into memory at once.



-- 
Steve
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Tutor] Example for read and readlines()

[Asad]

Hi All ,

 If I am loading a logfile what should I use from the option 1,2,3

f3 = open ( r"/a/b/c/d/test/test_2814__2018_10_05_12_12_45/logA.log", 'r' )

1) should only iterate over f3

2) st = f3.read()
Should iterate over st


3) st1 = f3.readlines()

Should iterate over st1

How are the above options different it they are not can there be some
examples to describe in which situations should we each method .


Thanks,

-- 
Asad Hasan
+91 9582111698
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor