Re: Not Responding When Dealing with Large Data
update_idletasks didn't work.
The code is this
file_path = filedialog.askopenfilename(filetypes=[('text files', '.txt')],
multiple=True, defaultextension=.txt)
for path in file_path:
fo = open(path, r)
for line in fo:
if myCase(line.lower()):
myList.append(line.lower())
fo.close()
def myCase(c):
if c in myList:
return False
if len(c) 8 or len(c) 80:
return False
return True
This processes a fair bit of data
On Thursday, June 19, 2014 12:06:26 AM UTC+1, Ian wrote:
On Wed, Jun 18, 2014 at 4:32 PM, cutey Love cuteywithl...@gmail.com wrote:
Hi, thanks for the replies,
I mean windows displays Not responding close the program now
How can I do it asynconistrically?
It's simple code just open file, loop through line by line and do some
comparons with the string.
If all you want is to prevent Windows from displaying that message
then I believe all you need to do is periodically call
frame.update_idletasks() (or the equivalent if you're using some
framework other than tkinter) so that the basic window events will get
processed.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On 2014-06-19 09:17, cutey Love wrote:
update_idletasks didn't work.
The code is this
file_path = filedialog.askopenfilename(filetypes=[('text files', '.txt')],
multiple=True, defaultextension=.txt)
for path in file_path:
fo = open(path, r)
for line in fo:
if myCase(line.lower()):
myList.append(line.lower())
fo.close()
def myCase(c):
if c in myList:
return False
if len(c) 8 or len(c) 80:
return False
return True
This processes a fair bit of data
It's quicker to look for something in a set than in a list, so if you
can use a set instead of a list, do so.
Also, checking the length of a string is quick, quicker than searching
a list.
Therefore, before processing the file, do:
mySet = set(myList)
and then you can say:
def myCase(c):
if len(c) 8 or len(c) 80:
return False
if c in mySet:
return False
return True
which can be shortened to:
def myCase(c):
return 8 = len(c) = 80 and c in mySet
--
https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On Thu, 19 Jun 2014 12:25:23 +0100, MRAB pyt...@mrabarnett.plus.com wrote: [snip] and then you can say: def myCase(c): if len(c) 8 or len(c) 80: return False if c in mySet: return False return True which can be shortened to: def myCase(c): return 8 = len(c) = 80 and c in mySet Don't you mean . . . return 8 = len(c) = 80 and c not in mySet ? -- To email me, substitute nowhere-spamcop, invalid-net. -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On 2014-06-19 17:21, Peter Pearson wrote: On Thu, 19 Jun 2014 12:25:23 +0100, MRAB pyt...@mrabarnett.plus.com wrote: [snip] and then you can say: def myCase(c): if len(c) 8 or len(c) 80: return False if c in mySet: return False return True which can be shortened to: def myCase(c): return 8 = len(c) = 80 and c in mySet Don't you mean . . . return 8 = len(c) = 80 and c not in mySet ? Yes, you're right. -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On Wed, Jun 18, 2014 at 1:20 PM, cutey Love cuteywithl...@gmail.com wrote: I'm trying to read in 10 lines of text, use some functions to edit them and then return a new list. The problem is my program always goes not responding when the amount of lines are a high number. I don't care how long the program takes to work, just need it to stop crashing? What are you doing with the data? Try reading in the file chunks at a time -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On Wednesday, June 18, 2014 1:20:13 PM UTC-4, cutey Love wrote: I'm trying to read in 10 lines of text, use some functions to edit them and then return a new list. The problem is my program always goes not responding when the amount of lines are a high number. I don't care how long the program takes to work, just need it to stop crashing? Please show some code. What kind of text (how long are the lines roughly)? thanks, --Tim -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
In 00d330e3-fc8a-4b7e-b2bd-f1a48bc33...@googlegroups.com cutey Love cuteywithl...@gmail.com writes: The problem is my program always goes not responding when the amount of lines are a high number. I don't care how long the program takes to work, just need it to stop crashing? How long have you tried waiting when the program stops responding? Perhaps the program just has a lot of work to do. You might try adding some print statements throughout the program, so it produces observable output instead of appearing to be unresponsive. Why do you say the program 'crashes'? Does it actually crash? -- John Gordon Imagine what it must be like for a real medical doctor to gor...@panix.comwatch 'House', or a real serial killer to watch 'Dexter'. -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On 06/18/2014 11:20 AM, cutey Love wrote: I'm trying to read in 10 lines of text, use some functions to edit them and then return a new list. The problem is my program always goes not responding when the amount of lines are a high number. I don't care how long the program takes to work, just need it to stop crashing? Welcome to the world of GUI programming. GUI programs are event-driven. When an event happens your code runs to handle the event. While you are handling an event, the rest of the GUI cannot run, until you return control to the main loop. So when you have to do long-running tasks, you need to find a way to return control to the main loop while your work continues. There are basically two ways to do this. One is to spawn a thread to do the work, the other is to use asynchronous programming, depending on what you need to get done. The problem with threads is that you cannot directly make GUI calls. Instead you have to set up a message queue of some sort, or maybe use flag variables, and then have a callback that fires periodically, regularly with a some kind of clock, or during the idle portion. Anyway your question is light on details, so my replay is rather light. Hopefully you are now pointed in the right direction for asking questions at least. A quick google search reveals this which is pretty much the sort of thing you wish to do: http://stackoverflow.com/quhttp://stackoverflow.com/questions/16745507/tkinter-how-to-use-threads-to-preventing-main-event-loop-from-freezingestions/16745507/tkinter-how-to-use-threads-to-preventing-main-event-loop-from-freezing -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On 6/18/14, 10:20 AM, cutey Love wrote: I'm trying to read in 10 lines of text, use some functions to edit them and then return a new list. How is it that you are opening the file, and iterating the contents? The problem is my program always goes not responding when the amount of lines are a high number. What do you mean, goes not responding? What environment are you running in? Windows and IDLE? I don't care how long the program takes to work, just need it to stop crashing? Note that not responding isn't the same as crashing. If your program crashed, it would likely print on your terminal something like: Traceback (most recent call last): ... lots of lines including useful information about the problem or Segmentation Fault If it just goes not responding for a while in your tight loop, it could simply mean just that: your IDE or whatever is focusing so hard on running your program that there aren't any cycles to say still here to the OS, so the OS has no clue what's going on and says to the user not responding. Have you waited to see if it ever completes? Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
Hi, thanks for the replies, I mean windows displays Not responding close the program now How can I do it asynconistrically? It's simple code just open file, loop through line by line and do some comparons with the string. On Wednesday, June 18, 2014 6:20:13 PM UTC+1, cutey Love wrote: I'm trying to read in 10 lines of text, use some functions to edit them and then return a new list. The problem is my program always goes not responding when the amount of lines are a high number. I don't care how long the program takes to work, just need it to stop crashing? -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On 6/18/14, 3:32 PM, cutey Love wrote: Hi, thanks for the replies, I mean windows displays Not responding close the program now How can I do it asynconistrically? It's simple code just open file, loop through line by line and do some comparons with the string. To get anything better from us than we've already given, you must tell us your environment and show us some code. Paul -- https://mail.python.org/mailman/listinfo/python-list
Re: Not Responding When Dealing with Large Data
On Wed, Jun 18, 2014 at 4:32 PM, cutey Love cuteywithl...@gmail.com wrote: Hi, thanks for the replies, I mean windows displays Not responding close the program now How can I do it asynconistrically? It's simple code just open file, loop through line by line and do some comparons with the string. If all you want is to prevent Windows from displaying that message then I believe all you need to do is periodically call frame.update_idletasks() (or the equivalent if you're using some framework other than tkinter) so that the basic window events will get processed. -- https://mail.python.org/mailman/listinfo/python-list
