[newbie] - python list into a sql query
Hi.
Can someone guide me into getting this to work? It's just really
querying a DB of an Autodiscovery tool to have a bunch of updated dns
files.
(Thought I'm still building the first script steps) I was able to
successfully query the DB against a single groupid, but am failing in
passing a list of integers into the sql query.
I'm failing miserably in,
sql = '''SELECT ip, host, dns FROM zabbix_tst_db.hosts WHERE hostid IN
(
select hostid from hosts_groups WHERE groupid IN (' +
','.join(map(str, %s)) + ')''' % grp_range
with
_mysql_exceptions.ProgrammingError: (1064, "You have an error in
your SQL syntax; check the manual that corresponds to your MySQL
server version for the right syntax to use near '' at line 1")
I'll gladly accept any other code change suggestion
#!/usr/bin/env python
import MySQLdb
import sys, os
import code
builder_path = '/opt/scripts/dns_builder'
grp_range = range(10,15)
try:
db = MySQLdb.connect(host="localhost",
user="tst",passwd="tst",db="tst_db" )
except MySQLdb.Error, e:
print "Error %d: %s" % (e.args[0], e.args[1])
cursor = db.cursor()
sql = '''SELECT ip, host, dns FROM zabbix_tst_db.hosts WHERE
hostid IN (
select hostid from hosts_groups WHERE groupid IN (' +
','.join(map(str, %s)) + ')''' % grp_range
cursor.execute(sql)
f = open('%s/dns_unknown_list.txt', 'w+') % builder_path
data = cursor.fetchall()
for row in data:
ip = row[0]
host = row[1]
dns = row[2]
if host == dns:
#tmn
if ip[0][:10] in ('140.254.30','10.13.74.')
group1_file = open('%s/group1.zone', 'w') % builder_path
print >>group1_file, '''$ORIGIN group1
'''
print >>group1_file, '%s IN A%s' % (dns, ip)
elif ip[0][:3] in ('8.128.46.','6.128.101')
group2_file = open('%s/group2.zone', 'w') % builder_path
print >>group2_file, '''$ORIGIN group2
'''
print >>group2_file, '%s IN A%s' % (dns, ip)
else:
print >>f, 'unknown IN A%s' % ip
db.close()
input_file.close()
f.close()
--
http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
On Jan 15, 4:46 pm, r0g wrote:
> João wrote:
> > On Jan 15, 2:38 pm, r0g wrote:
> >> João wrote:
> >>> On Jan 14, 5:58 pm, r0g wrote:
> >>>> João wrote:
> >>>>> On Jan 12, 10:07 pm, r0g wrote:
> >>>>>> João wrote:
> >>>>> for the following data,
> >>>>> authentication = "UID=somestring&"
> >>>>> message = 'PROBLEM severity High: OperatorX Plat1(locationY) global
> >>>>> Succ. : 94.47%'
> >>>>> dest_number = 'XXX'
> >>>>> url_values = urlencode({'M':message})
> >>>>> enc_data = authentication + url_values + dest_number
> >>>>> I'm getting null for
> >>>>> full_url = Request(url, enc_data, headers)
> >>>>> and thus,
> >>>>> response = urlopen(full_url).read()
> >>>>> returns,
> >>>>> TypeError:
> >>>>> )
> >>>> Are you sure it's returning a null and not just some other unexpected
> >>>> type?
> >>>> I think your problem may be that you are passing a urllib2 class to
> >>>> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g.
> >>>> import urllib2
> >>>> request_object = urllib2.Request('http://www.example.com')
> >>>> response = urllib2.urlopen(request_object)
> >>>> the_page = response.read()
> >>>> Roger.
> >>> Thanks Roger.
> >>> I think it's a null because i did a print(full_url) right after the
> >>> Request
> >>> I tried
> >>> request_object = urllib2.Request('http://www.example.com')
> >>> print(request_object)
> >>> but when printing I get:
> >> Hi João,
>
> >> That's exactly what you want, an object that is an instance of the
> >> Request class. That object doesn't do anything by itself, you still need
> >> to a) Connect to the server and request that URL and b) Read the data
> >> from the server.
>
> >> a) To connect to the web server and initialize the request you need to
> >> call urllib2.urlopen() with the Request object you just created and
> >> assign the result to a name e.g.
>
> >>>> response = urllib2.urlopen(request_object)
> >> That will give you an object (response) that you can call the .read()
> >> method of to get the web page data.
>
> >>>> the_page = response.read()
> >> If that doesn't make sense or seem to work for you then please try
> >> reading the following website from top to bottom before taking any
> >> further steps...
>
> >>http://www.voidspace.org.uk/python/articles/urllib2.shtml
>
> >>> I've read about Python 2.4 not playing well with proxies even with no
> >>> proxy activated.
> >>> Any sugestion?
> >> I doubt any language can play well with proxies if there are none so I
> >> doubt it's a factor ;)
>
> >> Good luck,
>
> >> Roger.
>
> > lol.
> > I've expressed myself poorly,
> > I meant I read about some issues when getting the Request + urlopen
> > working when there's a proxy involved (like in my case)
> > even when activating a no_proxy configuration, something like,
>
> > proxy_support = urllib.ProxyHandler({})
> > opener = urllib.build_opener(proxy_support)
> > urllib.install_opener(opener)
>
> > But I don't know how to use it :(
>
> That is how you use it IIRC, this installs the proxy handler into urllib
> and subsequent objects you subclass from urllib will use the custom handler.
>
> From what I can tell, you should be using urllib2 though, not urllib.
>
> Lets take a step back. You had the following line...
>
> request_object = urllib2.Request('http://www.example.com')
>
> ...You printed it and it showed that you had created a Request object
> right. Now what happens when you type...
>
> response = urllib2.urlopen(request_object)
> print response
>
> ?
>
> Roger.
Thanks for the patience Roger.
Your explanation opened my eyes.
I finally got it to work, and it turned out I didn't have to include
any custom proxy handler to avoid our proxy.
It ended on such a small and simple block of code after getting the
pieces together..
#!/usr/bin/env python
import sys
from urllib2 import Request, urlopen
from urllib import urlencode
authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&"
url = 'http://10.112.28.221:38080/GwHTTPin/sendtext'
encoded_data = urlencode({'M':sys.argv[2],
'N':sys.argv[1]})
# Was having problem with this one, shouldn't have tried to pass the
authentication as an urlencode parameter because it was breaking the
password string!
sent_data = authentication + encoded_data
full_url = Request(url,sent_data)
response = urlopen(full_url).read()
--
http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
On Jan 15, 4:46 pm, r0g wrote:
> João wrote:
> > On Jan 15, 2:38 pm, r0g wrote:
> >> João wrote:
> >>> On Jan 14, 5:58 pm, r0g wrote:
> >>>> João wrote:
> >>>>> On Jan 12, 10:07 pm, r0g wrote:
> >>>>>> João wrote:
> >>>>> for the following data,
> >>>>> authentication = "UID=somestring&"
> >>>>> message = 'PROBLEM severity High: OperatorX Plat1(locationY) global
> >>>>> Succ. : 94.47%'
> >>>>> dest_number = 'XXX'
> >>>>> url_values = urlencode({'M':message})
> >>>>> enc_data = authentication + url_values + dest_number
> >>>>> I'm getting null for
> >>>>> full_url = Request(url, enc_data, headers)
> >>>>> and thus,
> >>>>> response = urlopen(full_url).read()
> >>>>> returns,
> >>>>> TypeError:
> >>>>> )
> >>>> Are you sure it's returning a null and not just some other unexpected
> >>>> type?
> >>>> I think your problem may be that you are passing a urllib2 class to
> >>>> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g.
> >>>> import urllib2
> >>>> request_object = urllib2.Request('http://www.example.com')
> >>>> response = urllib2.urlopen(request_object)
> >>>> the_page = response.read()
> >>>> Roger.
> >>> Thanks Roger.
> >>> I think it's a null because i did a print(full_url) right after the
> >>> Request
> >>> I tried
> >>> request_object = urllib2.Request('http://www.example.com')
> >>> print(request_object)
> >>> but when printing I get:
> >> Hi João,
>
> >> That's exactly what you want, an object that is an instance of the
> >> Request class. That object doesn't do anything by itself, you still need
> >> to a) Connect to the server and request that URL and b) Read the data
> >> from the server.
>
> >> a) To connect to the web server and initialize the request you need to
> >> call urllib2.urlopen() with the Request object you just created and
> >> assign the result to a name e.g.
>
> >>>> response = urllib2.urlopen(request_object)
> >> That will give you an object (response) that you can call the .read()
> >> method of to get the web page data.
>
> >>>> the_page = response.read()
> >> If that doesn't make sense or seem to work for you then please try
> >> reading the following website from top to bottom before taking any
> >> further steps...
>
> >>http://www.voidspace.org.uk/python/articles/urllib2.shtml
>
> >>> I've read about Python 2.4 not playing well with proxies even with no
> >>> proxy activated.
> >>> Any sugestion?
> >> I doubt any language can play well with proxies if there are none so I
> >> doubt it's a factor ;)
>
> >> Good luck,
>
> >> Roger.
>
> > lol.
> > I've expressed myself poorly,
> > I meant I read about some issues when getting the Request + urlopen
> > working when there's a proxy involved (like in my case)
> > even when activating a no_proxy configuration, something like,
>
> > proxy_support = urllib.ProxyHandler({})
> > opener = urllib.build_opener(proxy_support)
> > urllib.install_opener(opener)
>
> > But I don't know how to use it :(
>
> That is how you use it IIRC, this installs the proxy handler into urllib
> and subsequent objects you subclass from urllib will use the custom handler.
>
> From what I can tell, you should be using urllib2 though, not urllib.
>
> Lets take a step back. You had the following line...
>
> request_object = urllib2.Request('http://www.example.com')
>
> ...You printed it and it showed that you had created a Request object
> right. Now what happens when you type...
>
> response = urllib2.urlopen(request_object)
> print response
>
> ?
>
> Roger.
Thanks for the patience Roger.
Your explanation opened my eyes.
I finally got it to work, and it turned out I didn't have to include
any custom proxy handler to avoid our proxy.
It ended on such a small and simple block of code after getting the
pieces together..
#!/usr/bin/env python
import sys
from urllib2 import Request, urlopen
from urllib import urlencode
authentication = "UID=124675&PW=gdberishyb8LcIANtvT89QVQ==&"
url = 'http://10.112.28.221:38080/GwHTTPin/sendtext'
encoded_data = urlencode({'M':sys.argv[2],
'N':sys.argv[1]})
# Was having problem with this one, shouldn't have tried to pass the
authentication as an urlencode parameter because it was breaking the
password string!
sent_data = authentication + encoded_data
full_url = Request(url,sent_data)
response = urlopen(full_url).read()
--
http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
EDIT: About the proxy. That's why I'm using the '-P' in the POST call. /usr/bin/POST -P -- http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
On Jan 15, 2:38 pm, r0g wrote:
> João wrote:
> > On Jan 14, 5:58 pm, r0g wrote:
> >> João wrote:
> >>> On Jan 12, 10:07 pm, r0g wrote:
> >>>> João wrote:
> >>> for the following data,
> >>> authentication = "UID=somestring&"
> >>> message = 'PROBLEM severity High: OperatorX Plat1(locationY) global
> >>> Succ. : 94.47%'
> >>> dest_number = 'XXX'
> >>> url_values = urlencode({'M':message})
> >>> enc_data = authentication + url_values + dest_number
> >>> I'm getting null for
> >>> full_url = Request(url, enc_data, headers)
> >>> and thus,
> >>> response = urlopen(full_url).read()
> >>> returns,
> >>> TypeError:
> >>> )
> >> Are you sure it's returning a null and not just some other unexpected
> >> type?
>
> >> I think your problem may be that you are passing a urllib2 class to
> >> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g.
>
> >> import urllib2
> >> request_object = urllib2.Request('http://www.example.com')
> >> response = urllib2.urlopen(request_object)
> >> the_page = response.read()
>
> >> Roger.
>
> > Thanks Roger.
> > I think it's a null because i did a print(full_url) right after the
> > Request
> > I tried
> > request_object = urllib2.Request('http://www.example.com')
> > print(request_object)
>
> > but when printing I get:
>
> Hi João,
>
> That's exactly what you want, an object that is an instance of the
> Request class. That object doesn't do anything by itself, you still need
> to a) Connect to the server and request that URL and b) Read the data
> from the server.
>
> a) To connect to the web server and initialize the request you need to
> call urllib2.urlopen() with the Request object you just created and
> assign the result to a name e.g.
>
> >> response = urllib2.urlopen(request_object)
>
> That will give you an object (response) that you can call the .read()
> method of to get the web page data.
>
> >> the_page = response.read()
>
> If that doesn't make sense or seem to work for you then please try
> reading the following website from top to bottom before taking any
> further steps...
>
> http://www.voidspace.org.uk/python/articles/urllib2.shtml
>
>
>
> > I've read about Python 2.4 not playing well with proxies even with no
> > proxy activated.
> > Any sugestion?
>
> I doubt any language can play well with proxies if there are none so I
> doubt it's a factor ;)
>
> Good luck,
>
> Roger.
lol.
I've expressed myself poorly,
I meant I read about some issues when getting the Request + urlopen
working when there's a proxy involved (like in my case)
even when activating a no_proxy configuration, something like,
proxy_support = urllib.ProxyHandler({})
opener = urllib.build_opener(proxy_support)
urllib.install_opener(opener)
But I don't know how to use it :(
--
http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
On Jan 14, 5:58 pm, r0g wrote:
> João wrote:
> > On Jan 12, 10:07 pm, r0g wrote:
> >> João wrote:
>
> > for the following data,
> > authentication = "UID=somestring&"
> > message = 'PROBLEM severity High: OperatorX Plat1(locationY) global
> > Succ. : 94.47%'
> > dest_number = 'XXX'
>
> > url_values = urlencode({'M':message})
> > enc_data = authentication + url_values + dest_number
>
> > I'm getting null for
> > full_url = Request(url, enc_data, headers)
>
> > and thus,
> > response = urlopen(full_url).read()
> > returns,
> > TypeError:
>
> > )
>
> Are you sure it's returning a null and not just some other unexpected
> type?
>
> I think your problem may be that you are passing a urllib2 class to
> urllib(1)'s urlopen. Try using urllib2's urlopen instead e.g.
>
> import urllib2
> request_object = urllib2.Request('http://www.example.com')
> response = urllib2.urlopen(request_object)
> the_page = response.read()
>
> Roger.
Thanks Roger.
I think it's a null because i did a print(full_url) right after the
Request
I tried
request_object = urllib2.Request('http://www.example.com')
print(request_object)
but when printing I get:
I've read about Python 2.4 not playing well with proxies even with no
proxy activated.
Any sugestion?
Thanks again
--
http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
On Jan 12, 10:07 pm, r0g wrote:
> João wrote:
> > On Jan 12, 8:05 pm, r0g wrote:
> >> João wrote:
> >>> Someone please?
> >> Haven't seen your original post yet mate, usenet can be flaky like that,
> >> might have been a good idea to quote your original post!
>
> >> Roger.
>
> > Thanks Roger.
>
> >> João wrote:
> >>> Someone please?
> >>> Hi.
> >>> I'm trying to figure out how to force URLencoding in my Python 2.4.3
> >>> environment, receiving data as an input argument but I'm really at a loss
> >>> here.
>
> >>> What am I doing wrong?
> > headers = { ’User-Agent’ : user_agent }
>
> Those quotes need to be either " or ' i.e.
>
> headers = { 'User-Agent' : user_agent }
>
> If that doesn't sort it then it would be helpful to see a traceback or,
> if it is not crashing, get a description of how it is failing to do what
> you expect.
>
> Cheers,
>
> Roger.
Weird, I had the correct quotes in my script, maybe I've pasted with
some error.
This is my current script,
(and though I got it working with the subprocess.call I don't know how
to use a pure Python version.
for the following data,
authentication = "UID=somestring&"
message = 'PROBLEM severity High: OperatorX Plat1(locationY) global
Succ. : 94.47%'
dest_number = 'XXX'
url_values = urlencode({'M':message})
enc_data = authentication + url_values + dest_number
I'm getting null for
full_url = Request(url, enc_data, headers)
and thus,
response = urlopen(full_url).read()
returns,
TypeError:
)
#!/usr/bin/env python
import sys
import os
import subprocess
from urllib import urlencode, urlopen
from urllib2 import Request
destination = sys.argv[1]
msg = sys.argv[2]
authentication = "UID=somestring&"
dest_number = "&N=%s" % destination
message = '%s' % msg
url_values = urlencode({'M':message})
enc_data = authentication + url_values + dest_number
url = 'http://10.112.28.221:38080/GwHTTPin/sendtext'
subprocess.call('echo "%s" | /usr/bin/POST -P "%s"' % (enc_data, url),
shell=True)
--
http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
On Jan 12, 8:05 pm, r0g wrote:
> João wrote:
> > Someone please?
>
> Haven't seen your original post yet mate, usenet can be flaky like that,
> might have been a good idea to quote your original post!
>
> Roger.
Thanks Roger.
> João wrote:
> > Someone please?
> > Hi.
> > I'm trying to figure out how to force URLencoding in my Python 2.4.3
> > environment, receiving data as an input argument but I'm really at a loss
> > here.
> > What am I doing wrong?
#!/usr/bin/env python
import sys
from urllib import urlencode, urlopen
from urllib2 import Request
import urlparse
destination = sys.argv[1]
msg = sys.argv[2] #Will I have problems with this one if the input is
multiline?
# the browser identifies itself using the User-Agent header
# after creating the Request object, it's possible to pass in a
dictionary of headers
user_agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;
SV1; .NET CLR 1.1.4322)' # force the request to be identified as IE
5.5
headers = { ’User-Agent’ : user_agent }
# force no proxy
authentication = 'UID=22541&PW=gdyb21LQTcIANtvYMT7QVQ==&'
# force Unicode display format
message = u'M=%s&' % msg
dest_number = 'N=%s' % destination
data = authentication + message + dest_number
url = 'http://10.112.28.221:38080/GwHTTPin/sendtext'
print 'Encoded URL:', url
#get full URL adding ? to it, followed by the encoded values
#full_url = url + '?' url_values
#should I force url_values = urllib.urlencode(data) instead?
full_url = urllib2.Request(url, data, headers)
response = urllib2.urlopen(full_url) #.urlopen works transparently
with proxies which do not require authentication
processed = urllib.open(full_url)
--
http://mail.python.org/mailman/listinfo/python-list
Re: force URLencoding script
Someone please? -- http://mail.python.org/mailman/listinfo/python-list
force URLencoding script
Hi.
I'm trying to figure out how to force URLencoding in my Python 2.4.3
environment receiving data an input argument but I'm really at a loss
here.
What am I doing wrong?
#!/usr/bin/env python
import sys
from urllib import urlencode, urlopen
from urllib2 import Request
import urlparse
destination = sys.argv[1]
msg = sys.argv[2] #Will I have problems with this one if the input is
multiline?
# the browser identifies itself using the User-Agent header
# after creating the Request object, it's possible to pass in a
dictionary of headers
user_agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;
SV1; .NET CLR 1.1.4322)' # force the request to be identified as IE
5.5
headers = { ’User-Agent’ : user_agent }
# force no proxy
authentication = 'UID=22541&PW=gdyb21LQTcIANtvYMT7QVQ==&'
# force Unicode display format
message = u'M=%s&' % msg
dest_number = 'N=%s' % destination
data = authentication + message + dest_number
url = 'http://10.112.28.221:38080/GwHTTPin/sendtext'
print 'Encoded URL:', url
#get full URL adding ? to it, followed by the encoded values
#full_url = url + '?' url_values
#should I force url_values = urllib.urlencode(data) instead?
full_url = urllib2.Request(url, data, headers)
response = urllib2.urlopen(full_url) #.urlopen works transparently
with proxies which do not require authentication
processed = urllib.open(full_url)
--
http://mail.python.org/mailman/listinfo/python-list
Re: plain text parsing to html (newbie problem)
Lie Ryan wrote:
> You can set MIME type and encoding from the MIME constructor
> email.mime.Text.MIMEText("Bold Text", "html", "utf-8")
>
> are you importing "import mime" or "import email.mime" or "import
> email.MIMEMultipart"?
Hi Lie.
I was importing as,
'from email.mime.text import MIMEText' which was returning that error
because I don't have
a mime subdir in my /usr/lib64/python2.4/email/ as I was expecting.
'import email.MIMEMultipart' worked perfectly (the mime methods are in
the /email/ root).
I'll try to redo my script right after I get some urgent things done.
Thanks :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: plain text parsing to html (newbie problem)
On Dec 10, 7:55 pm, Lie Ryan wrote: > > and, is there any reason why you're not using the email and > smtplib?http://docs.python.org/library/email-examples.html Mainly because I was unaware of them :( I just read about them and I found all the Subject, From, To classes, but what about Content-Type? I need Content-Type: text/html and charset="utf-8" so that I can pass a basic CSS formatting. How can I do that? With email.mime.base.MIMEBase? I'll have to update that in my code later, as I already got it to work and my current RHEL5.3 default Python 2.4.3 (#1, Sep 17 2008) install is complaining with "ImportError: No module named mime.multipart" Thank you for your patience and feedback -- http://mail.python.org/mailman/listinfo/python-list
Re: plain text parsing to html (newbie problem)
On Dec 10, 7:55 pm, Lie Ryan wrote: > > and, is there any reason why you're not using the email and > smtplib?http://docs.python.org/library/email-examples.html Mainly because I was unaware of them :( I just read about them and I found all the Subject, From, To classes, but what about Content-Type? I need Content-Type: text/html and charset="utf-8" so that I can pass a basic CSS formatting. How can I do that? With email.mime.base.MIMEBase? I'll have to update that in my code later, as I already got it to work and my current RHEL5.3 default Python 2.4.3 (#1, Sep 17 2008) install is complaining with "ImportError: No module named mime.multipart" Thank you for your patience and feedback -- http://mail.python.org/mailman/listinfo/python-list
Re: plain text parsing to html (newbie problem)
Thanks for the output. akean, I've installed ipython and I'm exploring it. Thanks. Terry, from what I've read stringIO allows us to store strings in a 'virtual' file. Can you please write just 2 lines exemplifying a write to and a read from an OS level file? MRAB, that 'mail' object should've been the os level 'mail' file. But as I'm a beginner I was using it wrong.. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: plain text parsing to html (newbie problem)
Thanks for the output. akean, I've installed ipython and I'm exploring it. Thanks. Terry, from what I've read stringIO allows us to store strings in a 'virtual' file. Can you please write just 2 lines exemplifying a write to and a read from an OS level file? MRAB, that 'mail' object should've been the os level 'mail' file. But as I'm a beginner I was using it wrong.. Thanks -- http://mail.python.org/mailman/listinfo/python-list
plain text parsing to html (newbie problem)
I apologize for my newbiness but I'm banging my head making this work :
(
What change must I made for the tag enforcement being reflected to the
'mail' file? Am I using the WritableObject class correctly?
(I'm getting a blank 'mail' file after running the .py script)
How can I see the output run in debug mode like in perl?
#!/usr/bin/env python
import sys, os
import subprocess
import re
import datetime
# simple class with write method to redirect print output to a file
class WritableObject:
def __init__(self):
self.content = []
def write(self, string):
self.content.append(string)
x=datetime.date.today()
, dd = x.year, x.day
mail = WritableObject()
print >>mail, "To: %s" % (mail1)
print >>mail, "Subject: %s day %s " % (, dd)
print >>mail, "Content-Type: text/html; charset=\"us-ascii\""
subprocess.call("php $HOME/report.php > temp.txt", shell=True)
body = subprocess.call("cat $HOME/.txt", shell=True)
print >>mail, ""
print >>mail, ""
print >>mail, ""
#copies common header in the Problem html to the OK one
subprocess.call("cp $HOME/mail $HOME/OK.html", shell=True)
input_file = open('mail', 'r+')
lines = input_file.readlines()
input_file.close()
output_file = open("mail", "w+")
for line in lines:
if line.startswith(''):
line = '' + line.rstrip('\n') + '' + '\n'
output_file.write(line)
if not body:
print >>mail, ''' [No problems detected ]. All monitored
metrics
OK
'''
else:
print >>mail, '''
Table
Table2
Table3
%s
''' % (body)
subprocess.call("sendmail m...@my.mail < mail", shell=True)
output_file.close()
--
http://mail.python.org/mailman/listinfo/python-list
Re: ISO library ref in printed form
kj wrote: Does anyone know where I can buy the Python library reference in printed form? (I'd rather not print the whole 1200+-page tome myself.) I'm interested in both/either 2.6 and 3.0. TIA! kj Why not download the documentation, take it to a local copy shop and have it printed and bound there? http://docs.python.org/download.html http://docs.python.org/3.1/download.html -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Lawrence D'Oliveiro wrote: In message , João Valverde wrote: Simple example usage case: Insert string into data structure in sorted order if it doesn't exist, else retrieve it. the_set = set( ... ) if str in the_set : ... "retrieval" case ... else : the_set.add(str) #end if Want sorted order? sorted(tuple(the_set)) What could be simpler? Try putting that inside a loop with thousands of iterations and you'll see what the problem is. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
alex23 wrote: João Valverde wrote: Currently I don't have a strong need for this. And clearly neither has anyone else, hence the absence from the stdlib. As others have pointed out, there are alternative approaches, and plenty of recipes on ActiveState, which seem to have scratched whatever itch there is for the data structure you're advocating. I can't resist quoting Sedgewick here. Then I'll shut up about it. [quote=http://www.cs.princeton.edu/~rs/talks/LLRB/LLRB.pdf] Abstract The red-black tree model for implementing balanced search trees, introduced by Guibas and Sedge- wick thirty years ago, is now found throughout our computational infrastructure. Red-black trees are described in standard textbooks and are the underlying data structure for symbol-table imple- mentations within C++, Java, Python, BSD Unix, and many other modern systems. [/quote] You'd think so, but no. You should correct him that in Python a balanced search tree is the useless cross between a dict and a database. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Paul Rubin wrote: João Valverde writes: Interesting, thanks. The concept is not difficult to understand but I'm not sure it would be preferable. A copy operation should have the same cost as a "snapshot", You mean a deep-copy? That is unnecessarily expensive; with a functional structure you can snapshot (or copy) by copying a single pointer. Shallow copy... undo is kind of redundant and multithreading... don't see a compelling use that would justify it. Here is one: http://groups.google.com/group/comp.lang.python/msg/1fbe66701e4bc65b I just skimmed that but if someone really needs multithreading for such intensive processing without wanting a database, fair enough I guess. Have you considered how the syntax would work in Python by the way? This: new_tree = old_tree.insert(object) Just looks wrong. It looks fine to me. Obviously you could support a wrapper with a mutating slot that holds a pointer to the tree. I didn't get the last part, sorry. But I think you'd have a lot of users annoyed that the interface is similar to a list yet their objects mysteriously disappear. To me, tree.insert() implies mutability but I defer that to others like yourself with more experience in Python than me. Rereading this I got what you meant by "wrapper with mutating slot". But that is (like I think you implied) functionally equivalent to a mutating data structure, with worse performance because of additional memory allocation and such. Is it faster to rebalance the tree with a persistent data structure? -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Paul Rubin wrote: João Valverde writes: Interesting, thanks. The concept is not difficult to understand but I'm not sure it would be preferable. A copy operation should have the same cost as a "snapshot", You mean a deep-copy? That is unnecessarily expensive; with a functional structure you can snapshot (or copy) by copying a single pointer. Shallow copy... Actually I meant whatever that snapshot operation is, minus the insertion, if that makes sense. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Paul Rubin wrote: João Valverde writes: Interesting, thanks. The concept is not difficult to understand but I'm not sure it would be preferable. A copy operation should have the same cost as a "snapshot", You mean a deep-copy? That is unnecessarily expensive; with a functional structure you can snapshot (or copy) by copying a single pointer. Shallow copy... undo is kind of redundant and multithreading... don't see a compelling use that would justify it. Here is one: http://groups.google.com/group/comp.lang.python/msg/1fbe66701e4bc65b I just skimmed that but if someone really needs multithreading for such intensive processing without wanting a database, fair enough I guess. Have you considered how the syntax would work in Python by the way? This: new_tree = old_tree.insert(object) Just looks wrong. It looks fine to me. Obviously you could support a wrapper with a mutating slot that holds a pointer to the tree. I didn't get the last part, sorry. But I think you'd have a lot of users annoyed that the interface is similar to a list yet their objects mysteriously disappear. To me, tree.insert() implies mutability but I defer that to others like yourself with more experience in Python than me. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Paul Rubin wrote: João Valverde writes: Could you clarify what you mean by immutable? As in... not mutable? As in without supporting insertions and deletions? Correct. That's has the same performance as using binary search on a sorted list. What's the point of using a tree for that? The idea is you can accomplish the equivalent of insertion or deletion by allocating a new root, along with the path down to the place you want to insert, i.e. O(log n) operations. So instead of mutating an existing tree, you create a new tree that shares most of its structure with the old tree, and switch over to using the new tree. This trivially lets you maintain snapshots of old versions of the tree, implement an "undo" operation, have a background thread do a complex operation on a snapshot while the foreground thread does any number of update-and-replace operations, etc. Interesting, thanks. The concept is not difficult to understand but I'm not sure it would be preferable. A copy operation should have the same cost as a "snapshot", undo is kind of redundant and multithreading... don't see a compelling use that would justify it. Also the interface is a mapping so it'd be rather nice to emulate dict's. Have you considered how the syntax would work in Python by the way? This: new_tree = old_tree.insert(object) Heh, that's a poor example for a mapping. But: bst[key] = object is even dicier for immutable structures no? -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Paul Rubin wrote: João Valverde writes: Could you clarify what you mean by immutable? As in... not mutable? As in without supporting insertions and deletions? Correct. That's has the same performance as using binary search on a sorted list. What's the point of using a tree for that? The idea is you can accomplish the equivalent of insertion or deletion by allocating a new root, along with the path down to the place you want to insert, i.e. O(log n) operations. So instead of mutating an existing tree, you create a new tree that shares most of its structure with the old tree, and switch over to using the new tree. This trivially lets you maintain snapshots of old versions of the tree, implement an "undo" operation, have a background thread do a complex operation on a snapshot while the foreground thread does any number of update-and-replace operations, etc. Interesting, thanks. The concept is not difficult to understand but I'm not sure it would be preferable. A copy operation should have the same cost as a "snapshot", undo is kind of redundant and multithreading... don't see a compelling use that would justify it. Also the interface is a mapping so it'd be rather nice to emulate dict's. Have you considered how the syntax would work in Python by the way? This: new_tree = old_tree.insert(object) Just looks wrong. The interface should support non functional idioms too. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Paul Rubin wrote: a...@pythoncraft.com (Aahz) writes: (In particular, WRT the bisect module, although insertion and deletion are O(N), the constant factor for doing a simple memory move at C speed swamps bytecode until N gets very large -- and we already have collections.deque() for some other common use cases.) Again, at least in my case, I'd hope for an immutable structure. Could you clarify what you mean by immutable? As in... not mutable? As in without supporting insertions and deletions? That's has the same performance as using binary search on a sorted list. What's the point of using a tree for that? -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Miles Kaufmann wrote: João Valverde wrote: To answer the question of what I need the BSTs for, without getting into too many boring details it is to merge and sort IP blocklists, that is, large datasets of ranges in the form of (IP address, IP address, string). Originally I was also serializing them in a binary format (but no more after a redesign). I kept the "merge and sort" part as a helper script, but that is considerably simpler to implement. ... As an anecdotal data point (honestly not trying to raise the "Python is slow" strawman), I implemented the same algorithm in C and Python, using pyavl. Round numbers were 4 mins vs 4 seconds, against Python (plus pyavl). Even considering I'm a worse Python programmer than C programmer, it's a lot. I know many will probably think I tried to do "C in Python" but that's not the case, at least I don' t think so. Anyway like I said, not really relevant to this discussion. What format were you using to represent the IP addresses? (Is it a Python class?) And why wouldn't you use a network address/subnet mask pair to represent block ranges? (It seems like being able to represent ranges that don't fit into a subnet's 2^n block wouldn't be that common of an occurrence, and that it might be more useful to make those ranges easier to manipulate.) I was using a bytes subclass. I'm not free to choose CIDR notation, range boundaries must be arbitrary. One of the major disadvantages of using a tree container is that usually multiple comparisons must be done for every tree operation. When that comparison involves a call into Python bytecode (for custom cmp/lt methods) the cost can be substantial. Compare that to Python's hash-based containers, which only need to call comparison methods in the event of hash collisions (and that's hash collisions, not hash table bucket collisions, since the containers cache each object's hash value). I would imagine that tree-based containers would only be worth using with objects with comparison methods implemented in C. I would flip your statement and say one of the advantages of using trees is that they efficiently keep random input sorted. Obviously no algorithm can do that with single comparisons. And not requiring a hash function is a desirable quality for non-hashable objects. There's a world beyond dicts. :) I profiled the code and indeed the comparisons dominated the execution time. Trimming the comparison function to the bare minimum, a single python operation, almost doubled the program's speed. Not that I'm trying to be an apologist, or reject your arguments; I can definitely see the use case for a well-implemented, fast tree-based container for Python. And so much the better if, when you need one, there was a clear consensus about what package to use (like PIL for image manipulation--it won't meet every need, and there are others out there, but it's usually the first recommended), rather than having to search out and evaluate a dozen different ones. Thanks, and I'm not trying to start a religion either. ;) -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
alex23 wrote: João Valverde wrote: Currently I don't have a strong need for this. And clearly neither has anyone else, hence the absence from the stdlib. As others have pointed out, there are alternative approaches, and plenty of recipes on ActiveState, which seem to have scratched whatever itch there is for the data structure you're advocating. Propose such alternative then. There are none that offer the same performance. At best they're workarounds. I don't care about recipes. That's called research. If people don't find it to be useful, that's fine. Surprising, but fine. And I don't have a need because I'm not using Python for my project. If I wanted to I couldn't, without implementing myself or porting to Python 3 a basic computer science data structure. While Python's motto is "batteries included" I've always felt there was an implicit "but not the kitchen sink" following it. Just because something "could" be useful shouldn't be grounds for inclusion. That's what pypi & the recipes are for. Ideally, little should be created wholesale for the stdlib, what should be added are the existing 3rd party modules that have become so ubiquitous that their presence on any python platform is just expected. Agreed. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Aahz wrote: In article , =?ISO-8859-1?Q?Jo=E3o_Valverde?= wrote: Anyway, I'm *not* trying to discourage you, just explain some of the roadblocks to acceptance that likely are why it hasn't already happened. If you're serious about pushing this through, you have two options: * Write the code and corresponding PEP yourself (which leads to the second option, anyway) * Lobby on the python-ideas mailing list Currently I don't have a strong need for this. I just believe it would be a benefit to a language I like a lot. Lobbying isn't my thing. I'd rather write code, but neither am I the most qualified person for the job. It would certainly be interesting and fun and challenging in a good way and a great way to learn some new stuff. But I would definitely need mentoring or asking some silly questions on the mailing list. Maybe I'll seriously consider it some other time. There's also another issue raise by Paul Rubin I wasn't even aware of, that the LGPL is not suitable for the standard library. Having to write a complete BST implementation in C is a drag. There are already good C libraries available. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Aahz wrote: In article , =?ISO-8859-1?Q?Jo=E3o_Valverde?= wrote: What's lacking is an associative array that preserves ordering, doesn't require a hash function and has fast insertions and deletions in O(log(n)). The particular algorithm to achieve this is a secondary issue. It's a BST for sure, AVL vs RBT vs something else. It's my fault for having opened the topic with simply "trees" instead, it would have avoided this vagueness problem, but I'm genuinely surprised to know there are no data structures that efficiently support such a common need in Python. And I really enjoy the using this language. Why AVL/RBT instead of B*? It's not that simple Another problem is that unless the tree is coded in C, the constant factors are going to swamp algorithmic complexity for many use cases -- that in turn makes it more difficult to deploy a PyPI library for real-world testing. I wouldn't consider anything other than C for such a module on efficiency alone, unless it was a prototype of course. But I have little knowledge about the Python C API. About B* trees, again not an expert but I don't see how the added complexity brings any noticeable advantage to implement the associative array data structure I mentioned. Simple is better. Anyway, I'm *not* trying to discourage you, just explain some of the roadblocks to acceptance that likely are why it hasn't already happened. If you're serious about pushing this through, you have two options: * Write the code and corresponding PEP yourself (which leads to the second option, anyway) * Lobby on the python-ideas mailing list Currently I don't have a strong need for this. I just believe it would be a benefit to a language I like a lot. Lobbying isn't my thing. I'd rather write code, but neither am I the most qualified person for the job. It would certainly be interesting and fun and challenging in a good way and a great way to learn some new stuff. But I would definitely need mentoring or asking some silly questions on the mailing list. Maybe I'll seriously consider it some other time. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: greg wrote: João Valverde wrote: What's lacking is an associative array that preserves ordering, doesn't require a hash function and has fast insertions and deletions in O(log(n)). Careful here -- you can't get away from the need for hashability just by using a tree. Even if you don't need to actually hash the values, it's still important that the criterion for ordering between objects doesn't change while they're in the tree, otherwise they'll be in the wrong place and won't be found by subsequent lookups. I'm aware :) Technically it's necessary to define a total ordering on the set of keys. > I'm genuinely surprised to know there are no data structures that efficiently support such a common need in Python. Is it really all that common? If it truly were common, there probably *would* be something for it in the stdlib by now. Obviously my experience differs, but those were my expectations. What sort of things are you doing that you want such a structure for? Maybe we can suggest a way of using the existing data structures to achieve the same goal. To answer the question of what I need the BSTs for, without getting into too many boring details it is to merge and sort IP blocklists, that is, large datasets of ranges in the form of (IP address, IP address, string). Originally I was also serializing them in a binary format (but no more after a redesign). I kept the "merge and sort" part as a helper script, but that is considerably simpler to implement. Crap, this sentence is totally confusing. I meant kept the merge code as a helper script and moved the rest to C, see next paragraph. Please note that I'm happy with my code, it works well. I intended to implement it in C all along (for a system daemon), even before trying Python. The python code was a side thing for testing/curiosity/fun. It prompted my original question but that was really about Python and the standard library itself, and I don't wish to waste anyone's time. As an anecdotal data point (honestly not trying to raise the "Python is slow" strawman), I implemented the same algorithm in C and Python, using pyavl. Round numbers were 4 mins vs 4 seconds, against Python (plus pyavl). Even considering I'm a worse Python programmer than C programmer, it's a lot. I know many will probably think I tried to do "C in Python" but that's not the case, at least I don' t think so. Anyway like I said, not really relevant to this discussion. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
greg wrote: João Valverde wrote: What's lacking is an associative array that preserves ordering, doesn't require a hash function and has fast insertions and deletions in O(log(n)). Careful here -- you can't get away from the need for hashability just by using a tree. Even if you don't need to actually hash the values, it's still important that the criterion for ordering between objects doesn't change while they're in the tree, otherwise they'll be in the wrong place and won't be found by subsequent lookups. I'm aware :) Technically it's necessary to define a total ordering on the set of keys. > I'm genuinely surprised to know there are no data structures that efficiently support such a common need in Python. Is it really all that common? If it truly were common, there probably *would* be something for it in the stdlib by now. Obviously my experience differs, but those were my expectations. What sort of things are you doing that you want such a structure for? Maybe we can suggest a way of using the existing data structures to achieve the same goal. To answer the question of what I need the BSTs for, without getting into too many boring details it is to merge and sort IP blocklists, that is, large datasets of ranges in the form of (IP address, IP address, string). Originally I was also serializing them in a binary format (but no more after a redesign). I kept the "merge and sort" part as a helper script, but that is considerably simpler to implement. Please note that I'm happy with my code, it works well. I intended to implement it in C all along, even before trying Python. The python code was a side thing for testing/curiosity/fun. It prompted my original question but that was really about Python and the standard library itself, and I don't wish to waste anyone's time. As an anecdotal data point (honestly not trying to raise the "Python is slow" strawman), I implemented the same algorithm in C and Python, using pyavl. Round numbers were 4 mins vs 4 seconds, against Python (plus pyavl). Even considering I'm a worse Python programmer than C programmer, it's a lot. I know many will probably think I tried to do "C in Python" but that's not the case, at least I don' t think so. Anyway like I said, not really relevant to this discussion. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Aahz wrote: In article <006078f0$0$9721$c3e8...@news.astraweb.com>, Steven D'Aprano wrote: Hash tables (dicts) are useful for many of the same things that trees are useful for, but they are different data structures with different strengths and weaknesses, and different uses. To argue that we don't need trees because we have dicts makes as much sense as arguing that we don't need dicts because we have lists. The problem is that trees are like standards: there are so many to choose from. Each has its own tradeoffs, and because Python dicts and lists can substitute for many of the traditionals uses of trees, the tradeoffs are less clear. I think nobody would object to adding trees to the standard library, but it will certainly require a clear PEP, preferably with a pointer to an existing PyPI library that has acquired real-world use. Wish I had asked this before this year's GSoC started. What's lacking is an associative array that preserves ordering, doesn't require a hash function and has fast insertions and deletions in O(log(n)). The particular algorithm to achieve this is a secondary issue. It's a BST for sure, AVL vs RBT vs something else. It's my fault for having opened the topic with simply "trees" instead, it would have avoided this vagueness problem, but I'm genuinely surprised to know there are no data structures that efficiently support such a common need in Python. And I really enjoy the using this language. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: João Valverde wrote: Aahz wrote: In article , Tom Reed wrote: Why no trees in the standard library, if not as a built in? I searched the archive but couldn't find a relevant discussion. Seems like a glaring omission considering the batteries included philosophy, particularly balanced binary search trees. No interest, no good implementations, something other reason? Seems like a good fit for the collections module. Can anyone shed some light? What do you want such a tree for? Why are dicts and the bisect module inadequate? Note that there are plenty of different tree implementations available from either PyPI or the Cookbook. A hash table is very different to a BST. They are both useful. The bisect module I'm not familiar with, I'll have to look into that, thanks. I have found pyavl on the web, it does the job ok, but there no implementations for python3 that I know of. Simple example usage case: Insert string into data structure in sorted order if it doesn't exist, else retrieve it. After browsing the bisect module I don't think it is the complete answer. Please correct me if I'm mistaken but... Ignoring for a moment that subjectively I feel this is not very pythonic for my use case, if I get back the insertion position, doesn't that mean I have to go over on average N/2 items on a linked list to insert the item in position? Maybe less for sophisticated implementations but still O(n)? That doesn't compare favorably to the cost of (possibly) having to rebalance a tree on insertion. I was indeed corrected on this shameful blunder, but in my defense array based lists are implementation dependent and the case for trees can be made on deletion. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Stefan Behnel wrote: João Valverde wrote: Besides some interface glitches, like returning None on delete if I recall correctly. That's actually not /that/ uncommon. Operations that change an object are not (side-effect free) functions, so it's just purity if they do not have a return value. Although practicality beats purity, sometimes... ;) Stefan I didn't know that. But in this case I think purity gets pummeled every time :) It's still not making sense to force a lookup to fetch something before deleting (another lookup operation). If that were imposed by python's internal machinery I'd suggest fixing that instead. To be clear what I mean by that is that it's just reference passing so it can't generate programmatic errors. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Stefan Behnel wrote: João Valverde wrote: Besides some interface glitches, like returning None on delete if I recall correctly. That's actually not /that/ uncommon. Operations that change an object are not (side-effect free) functions, so it's just purity if they do not have a return value. Although practicality beats purity, sometimes... ;) Stefan I didn't know that. But in this case I think purity gets pummeled every time :) It's still not making sense to force a lookup to fetch something before deleting (another lookup operation). If that were imposed by python's internal machinery I'd suggest fixing that instead. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Jason Scheirer wrote: On Jun 25, 10:32 pm, a...@pythoncraft.com (Aahz) wrote: In article , Tom Reed wrote: Why no trees in the standard library, if not as a built in? I searched the archive but couldn't find a relevant discussion. Seems like a glaring omission considering the batteries included philosophy, particularly balanced binary search trees. No interest, no good implementations, something other reason? Seems like a good fit for the collections module. Can anyone shed some light? What do you want such a tree for? Why are dicts and the bisect module inadequate? Note that there are plenty of different tree implementations available from either PyPI or the Cookbook. -- Aahz (a...@pythoncraft.com) <*>http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha ...And heapq is more-or-less an emulation of a tree structure in its underlying model. I once wrote a binary sorted tree data structure for Python in C. It performed anywhere from 15-40% worse than dicts. I think with optimization it will only perform 10% worse than dicts at best. Oh hey maybe that is why trees aren't an emphasized part of the standard. They are going to be much slower than the ultra-optimized dicts already in the standard lib. But a dict can't be used to implement a (sorted) table ADT. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Aahz wrote: In article , Tom Reed wrote: Why no trees in the standard library, if not as a built in? I searched the archive but couldn't find a relevant discussion. Seems like a glaring omission considering the batteries included philosophy, particularly balanced binary search trees. No interest, no good implementations, something other reason? Seems like a good fit for the collections module. Can anyone shed some light? What do you want such a tree for? Why are dicts and the bisect module inadequate? Note that there are plenty of different tree implementations available from either PyPI or the Cookbook. A hash table is very different to a BST. They are both useful. The bisect module I'm not familiar with, I'll have to look into that, thanks. I have found pyavl on the web, it does the job ok, but there no implementations for python3 that I know of. Simple example usage case: Insert string into data structure in sorted order if it doesn't exist, else retrieve it. After browsing the bisect module I don't think it is the complete answer. Please correct me if I'm mistaken but... Ignoring for a moment that subjectively I feel this is not very pythonic for my use case, if I get back the insertion position, doesn't that mean I have to go over on average N/2 items on a linked list to insert the item in position? Maybe less for sophisticated implementations but still O(n)? That doesn't compare favorably to the cost of (possibly) having to rebalance a tree on insertion. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Aahz wrote: In article , Tom Reed wrote: Why no trees in the standard library, if not as a built in? I searched the archive but couldn't find a relevant discussion. Seems like a glaring omission considering the batteries included philosophy, particularly balanced binary search trees. No interest, no good implementations, something other reason? Seems like a good fit for the collections module. Can anyone shed some light? What do you want such a tree for? Why are dicts and the bisect module inadequate? Note that there are plenty of different tree implementations available from either PyPI or the Cookbook. A hash table is very different to a BST. They are both useful. The bisect module I'm not familiar with, I'll have to look into that, thanks. I have found pyavl on the web, it does the job ok, but there no implementations for python3 that I know of. The main problem with pyavl by the way is that it doesn't seem to be subclassable (?). Besides some interface glitches, like returning None on delete if I recall correctly. There's also rbtree, which I didn't try. And I think that's it. On the whole not a lot of choice and not as practical for such a common data structure. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
João Valverde wrote: Aahz wrote: In article , Tom Reed wrote: Why no trees in the standard library, if not as a built in? I searched the archive but couldn't find a relevant discussion. Seems like a glaring omission considering the batteries included philosophy, particularly balanced binary search trees. No interest, no good implementations, something other reason? Seems like a good fit for the collections module. Can anyone shed some light? What do you want such a tree for? Why are dicts and the bisect module inadequate? Note that there are plenty of different tree implementations available from either PyPI or the Cookbook. A hash table is very different to a BST. They are both useful. The bisect module I'm not familiar with, I'll have to look into that, thanks. I have found pyavl on the web, it does the job ok, but there no implementations for python3 that I know of. Simple example usage case: Insert string into data structure in sorted order if it doesn't exist, else retrieve it. Crap, sorry about the mixed identities, I'm not using my own machine. The original post is mine also. -- http://mail.python.org/mailman/listinfo/python-list
Re: No trees in the stdlib?
Aahz wrote: In article , Tom Reed wrote: Why no trees in the standard library, if not as a built in? I searched the archive but couldn't find a relevant discussion. Seems like a glaring omission considering the batteries included philosophy, particularly balanced binary search trees. No interest, no good implementations, something other reason? Seems like a good fit for the collections module. Can anyone shed some light? What do you want such a tree for? Why are dicts and the bisect module inadequate? Note that there are plenty of different tree implementations available from either PyPI or the Cookbook. A hash table is very different to a BST. They are both useful. The bisect module I'm not familiar with, I'll have to look into that, thanks. I have found pyavl on the web, it does the job ok, but there no implementations for python3 that I know of. Simple example usage case: Insert string into data structure in sorted order if it doesn't exist, else retrieve it. -- http://mail.python.org/mailman/listinfo/python-list
Network game using mysql
Good evening, I am making an online game that stores its data in a mysql database. The thing is that I can't allow the players to interact directly with the important tables of database (they could cheat if I give them access) so I only allow them to write on a table named commands and then a python program running on the server will interpreter that command and update the important tables. For example, a user that wants to build an house writes on the commands table 'build house', and then my python program takes the resources of the user and gives him an house. The thing is that I need to have a program reading the command list in a infinite while loop and I don't know if there is a better way to do this, I am afraid that if the command list gets to crowed my python program may not handle all the commands in time. And if I put two programs reading the same command list sometimes they handle the command twice.. Hope you can help me, Thanks. _ Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy! http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us-- http://mail.python.org/mailman/listinfo/python-list
[no subject]
-- http://mail.python.org/mailman/listinfo/python-list
Orientation on MySql using python
Hello everyone, I want to make a program in python that uses Mysql aswell.. I don't know anything about mysql can you tell me some good books/tutorials where I can read something about mysql using python? Thanks. _ Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy! http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us-- http://mail.python.org/mailman/listinfo/python-list
mod_python + apache at windows
Hi... Ane one can give me an example to install mod_python in apache at windows? Hug, Mello. -- http://mail.python.org/mailman/listinfo/python-list
Re: Rationale for read-only property of co_code
On Apr 3, 4:43 am, Scott David Daniels <[EMAIL PROTECTED]> wrote: > Nope: If you change the code in-place, the whole stack's references > to where they were running would need to get updated to corresponding > locations in the new code. _That_ is a lot of work. Ah, there it is. Now I get it, it makes perfect sense. Looks like I'll have to stick to the usual mechanisms! Thanks everyone! --- João Neves -- http://mail.python.org/mailman/listinfo/python-list
Re: Rationale for read-only property of co_code
On 2 Abr, 21:38, "Chris Mellon" <[EMAIL PROTECTED]> wrote: > On Wed, Apr 2, 2008 at 2:33 PM, João Neves <[EMAIL PROTECTED]> wrote: > > On Apr 2, 5:41 pm, "Dan Upton" <[EMAIL PROTECTED]> wrote: > > > > The thing I've been wondering is why _is_ it read-only? In what > > > > circumstances having write access to co_code would break the language > > > > or do some other nasty stuff? > > > > > João Neves > > > > I can't speak to Python's implementation in particular, but > > > self-modifying code in general is unpleasant. It certainly is > > > possible to support it in runtime environments, but it's usually not > > > easy to do so. That is, of course, somewhat dependent on the > > > implementation of the runtime environment, and even to some degree the > > > underlying hardware. (For instance, the compiled code you want to run > > > could be in the hardware cache; if you then change the instructions at > > > those addresses in memory, it's not always straightforward to get the > > > processor to realize it needs to load the new data into the > > > instruction cache.) Plus, I suppose it might be possible to break > > > strong (even dynamic) typing if you start changing the code around > > > (although I can't construct an example off the top of my head). > > > Indeed, the caching issue is a relevant one I guess, and adding the > > mechanism to go around that might have a significant impact on > > performance. > > I haven't looked in detail into it, but I agree with your opinion > > concerning strong and dynamic typing. If type check is done while > > compiling to bytecode, there is no guarantee the modified bytecode > > will respect the rules, for instance. I don't know though, haven't > > really checked how it's done at this point. :) > > I will be fiddling around with the Python VM these days anyway, and > > since I'm going to muck around the bytecode, I might just try to see > > the effects of removing the read-only restriction from co_code. > > There is no need to overwrite co_code. Create a new code object with > your desired bytecode and use that instead. Yes, it may work (haven't tested - isn't there any problem with stuff like co_name, for instance?), but for simplicity's sake, wouldn't it be far more convenient if you could just write over co_code? :) In the end, it's all a matter of convenience, I guess. -- http://mail.python.org/mailman/listinfo/python-list
Re: Rationale for read-only property of co_code
On Apr 2, 5:41 pm, "Dan Upton" <[EMAIL PROTECTED]> wrote: > > The thing I've been wondering is why _is_ it read-only? In what > > circumstances having write access to co_code would break the language > > or do some other nasty stuff? > > > João Neves > > I can't speak to Python's implementation in particular, but > self-modifying code in general is unpleasant. It certainly is > possible to support it in runtime environments, but it's usually not > easy to do so. That is, of course, somewhat dependent on the > implementation of the runtime environment, and even to some degree the > underlying hardware. (For instance, the compiled code you want to run > could be in the hardware cache; if you then change the instructions at > those addresses in memory, it's not always straightforward to get the > processor to realize it needs to load the new data into the > instruction cache.) Plus, I suppose it might be possible to break > strong (even dynamic) typing if you start changing the code around > (although I can't construct an example off the top of my head). Indeed, the caching issue is a relevant one I guess, and adding the mechanism to go around that might have a significant impact on performance. I haven't looked in detail into it, but I agree with your opinion concerning strong and dynamic typing. If type check is done while compiling to bytecode, there is no guarantee the modified bytecode will respect the rules, for instance. I don't know though, haven't really checked how it's done at this point. :) I will be fiddling around with the Python VM these days anyway, and since I'm going to muck around the bytecode, I might just try to see the effects of removing the read-only restriction from co_code. > > In short, you need a good reason to support self-modifying code, and > my guess is nobody could come up with one for Python. > > -dan >From what I've seen, Python has some support for lambda expressions, albeit it may be limited. Self-modifying code would greatly enhance this feature, I guess. Well, at least from a Lisp newbie perspective. :) --- João Neves -- http://mail.python.org/mailman/listinfo/python-list
Re: Rationale for read-only property of co_code
On Apr 2, 4:35 pm, [EMAIL PROTECTED] wrote: > Are Python bytes codes Python byte codes? I'm not quite sure I understood your question, sorry. > Do you foresee any machine-dependent optimizations? In my personal case, I am not looking for optimizations in the generated bytecode. Let me give a very basic example. Say we have these two functions: def inc(x): x = x + 1 def dec(x): x = x - 1 Examining the compiled bytecodes for these two functions: >>> inc.func_code.co_code '|\x00\x00d\x01\x00\x17}\x00\x00d\x00\x00S' >>> dec.func_code.co_code '|\x00\x00d\x01\x00\x18}\x00\x00d\x00\x00S' Now suppose that I wanted to mess with inc, and have it behave like dec. For that, I would like to do something like this, for instance: >>> inc.func_code.co_code = '|\x00\x00d\x01\x00\x18}\x00\x00d\x00\x00S' Of course, as of this moment, I get a TypeError exception, because co_code is read-only. The thing I've been wondering is why _is_ it read-only? In what circumstances having write access to co_code would break the language or do some other nasty stuff? João Neves -- http://mail.python.org/mailman/listinfo/python-list
Re: wsdl (soap) without code generation
On Apr 2, 3:06 pm, Thomas Guettler <[EMAIL PROTECTED]> wrote: > Hi, > > I looked for a solution to talk to a web service which > offers its signature with a wsdl file. > > I googled for 'wsdl python' and found ZSI. > > This project uses code generation. That's something > I don't like. > > The book 'dive into python' uses SOAPpy. This looks > better since it does not generate source code. > But the last release and first release is from 2001. > > ZSI seems to have integrated SOAPpy. > > I am new to WSDL and SOAP. Do I need a WSDL parsing > routine at all? I guess my wsdl definition won't change > during the next years. So I could read the wsdl with > my eyes and make a fitting soap call... > > Any thoughts? > > -- > Thomas Guettler,http://www.thomas-guettler.de/ > E-Mail: guettli (*) thomas-guettler + de I've been using SOAPpy directly in one of my projects, and so far my experience couldn't be better. SOAPpy can handle (hand-written) WSDLs well enough for me, and so far it has given me no problems at all. -- http://mail.python.org/mailman/listinfo/python-list
Rationale for read-only property of co_code
Hello all, I've got this question that has been nagging me for a few days now. What are the reasons for us to have co_code as read-only? I've been trying to get some info about it, but I kept hitting the wall. Correct me if I'm wrong, but as far as I understand, co_code represents the compiled bytecode that should be run when, for instance, a function is called. Wouldn't it be beneficial for programmers to be able to change the bytecode in runtime? I mean, one can't, as far as I'm aware, change the bytecode by accident, so if the programmer would wish to change a function at runtime, he could do so at his own risk. If there is a higher reason behind the read-only property of co_code, I definitely fail to see it, and would like to know what it is. If not, why aren't we allowed to write into it? Thanks in advance, João Neves -- http://mail.python.org/mailman/listinfo/python-list
Checking processes running under Windows
Hello all! I'm trying to write a script that needs to check which processes
are running under Windows (XP Pro, Home, whatever). The method I'm using is:
>>> process_list = os.popen('TASKLIST').read()
However, XP Home doesn't have the tasklist.exe tool so, this is kind of
useless in that OS. Do you have any other methods I can use for this?
Thanks!
--
http://mail.python.org/mailman/listinfo/python-list
Webservices, SOAP and Python
Hello all, I'm trying to build a webserver client and I'm having quite the trouble. I ended up using ZSI (since it's the only one that I could actually install on my Linux system) but I wonder: is there any other software for this purpose? I read about SOAPpy and PyXML but since both are discontinued, what do you use? By the way, I'm having trouble connecting ZSI-using scripts to the internet. Every other script works. Why could this be happening? How can I prevent this? (I'm behind a proxy) Cheers! -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Web Programming - looking for examples of solid high-traffic sites
Please have a look at Plone and Zope. "During the month of January 2006, we've had approx. 167 million hits" plone.org On 2007-05-16 23:04:17 +0200, Victor Kryukov <[EMAIL PROTECTED]> said: > Hello list, > > our team is going to rewrite our existing web-site, which has a lot of > dynamic content and was quickly prototyped some time ago. > > Today, as we get better idea of what we need, we're going to re-write > everything from scratch. Python is an obvious candidate for our team: > everybody knows it, everybody likes it, it has *real* objects, nice > clean syntax etc. > > Our main requirement for tools we're going to use is rock-solid > stability. As one of our team-members puts it, "We want to use tools > that are stable, has many developer-years and thousands of user-years > behind them, and that we shouldn't worry about their _versions_." The > main reason for that is that we want to debug our own bugs, but not > the bugs in our tools. > > Our problem is - we yet have to find any example of high-traffic, > scalable web-site written entirely in Python. We know that YouTube is > a suspect, but we don't know what specific python web solution was > used there. > > TurboGears, Django and Pylons are all nice, and provides rich features > - probably too many for us - but, as far as we understand, they don't > satisfy the stability requirement - Pylons and Django hasn't even > reached 1.0 version yet. And their provide too thick layer - we want > something 'closer to metal', probably similar to web.py - > unfortunately, web.py doesn't satisfy the stability requirement > either, or so it seems. > > So the question is: what is a solid way to serve dynamic web pages in > python? Our initial though was something like python + mod_python + > Apache, but we're told that mod_python is 'scary and doesn't work very > well'. > > And although http://www.python.org/about/quotes/ lists many big names > and wonderful examples, be want more details. E.g. our understanding > is that Google uses python mostly for internal web-sites, and > performance is far from perfect their. YouTube is an interesting > example - anybody knows more details about that? > > Your suggestions and comments are highly welcome! > > Best Regards, > Victor. -- João Santos [EMAIL PROTECTED] www.thegoldenaura.com -- http://mail.python.org/mailman/listinfo/python-list
