Hello everyone,
I wrote a simple script to automate a task in python.
import datetime
import urllib2
import re
f1=open("url-list.txt","r")
f2=open("results.txt","w")
count=3680
print datetime.datetime.now()
f2.write(str(datetime.datetime.now())+"\n")
for line in f1.readlines():
if line!="URL\n":
count=count+1
print str(count)+":\t"+line.split("\n")[0]
resp_code=0
try:
response = urllib2.urlopen(line)
html=response.read()
resp_code=response.getcode()
if resp_code==200:
q=re.search("proptiger.com",html)
if q:
print "Yes "+str(datetime.datetime.now())
f2.write(str(count)+"\t"+line.split("\n")[0]+"\t"+"Yes\n")
else:
print "No "+str(datetime.datetime.now())
f2.write(str(count)+"\t"+line.split("\n")[0]+"\t"+"No\n")
else:
print resp_code+" "+str(datetime.datetime.now())
f2.write(str(count)+"\t"+line.split("\n")[0]+"\t"+str(resp_code)+"\n")
except:
print "Error"+str(datetime.datetime.now())
f2.write(str(count)+"\t"+line.split("\n")[0]+"\t"+"error\n")
print datetime.datetime.now()
f2.write(str(datetime.datetime.now()))
f1.close()
f2.close()
It works to an extent, as after some time, it starts to hang, bringing the
CPU all the way up, and not writing anything (or sometimes writing only
pieces of text) to the results.txt file.
could anyone tell what can be the problem?
--
--
Mailing list guidelines and other related articles: http://lug-iitd.org/Footer
---
You received this message because you are subscribed to the Google Groups
"Linux User Group @ IIT Delhi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
