Detecting open files and forcing closure
Hi Everyone, We have a process that does a copy of a share from one location to another. This usually works fine but can occasionally bomb if a file is opened by a user somewhere. Is there a way to code detection of open files and force a close? The files in question are typically PDF files if that matters. Any pointers you can provide on this would be greatly appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Detecting open files and forcing closure
MRAB wrote: Andrew Robert wrote: Hi Everyone, We have a process that does a copy of a share from one location to another. This usually works fine but can occasionally bomb if a file is opened by a user somewhere. Is there a way to code detection of open files and force a close? The files in question are typically PDF files if that matters. Any pointers you can provide on this would be greatly appreciated. If the file is open then an exception will be raised. You could catch it, sleep a while, and then retry, or continue with the other files and then come back to it and retry. It might take more than one retry. Anyway, it's a bad idea to force a close, even if that's possible. The usual scenario is that a user will leave a PDF open and then go home for the evening. They are simply viewing and not modifying the file. When the XCOPY executes, it signals a failure and subsequent scheduler job abend. What I need to do is detect if files are open for viewing and force a close before the copy operation is attempted. Sleeping and retrying the copy is not an option because the user will likely leave it open all night. Is there a way to detect the open files and close them out? -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie question: if var1 == var2:
Two issues regarding script.
You have a typo on the file you are trying to open.
It is listed with a file extension of .in when it should be .ini .
The next issue is that you are comparing what was read from the file
versus the variable.
The item read from file also contains and end-of-line character so
they will never match.
To get around this:
#!/usr/bin/python
fname = open("test43.ini")
var = 'tree'
for item in fname:
print "item: ", item,
if (item.rstrip("\n") == var):
print "found tree: ", item,
else:
print "No tree found"
David wrote:
Il Fri, 28 Nov 2008 19:47:01 -0800 (PST), [EMAIL PROTECTED] ha
scritto:
I dont understand why the following code never finds "tree".
New line marker to be stripped?
if item == var:
if item.strip() == var:
D.
--
http://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: question on python syntax
a.m. wrote: If I type this in shell $ ./yourfile.py 12:34 PM What does '$', '.', '/' and ' means in this succession? Note: 12:34 PM is a argument to the yourfile.py. This not python syntax but Unix shell. $ = shell prompt ./= look for the program in my current working directory yourfile.py = the program name 12:34 = argument 1 PM= argument 2 = run in background -- http://mail.python.org/mailman/listinfo/python-list
Re: Python editor
[EMAIL PROTECTED] wrote: John Salerno wrote: Is it possible to get vim-python for Windows, or is that just a Linux build? It builds for windows. When installed, you may also want to consider the python add-on located at http://www.vim.org/scripts/script.php?script_id=790 Enhanced version of the original (from vim6.1) python.vim for Python programming language. The changes since the original python.vim are: - changed strings highlighting; - enhanced special symbols highlighting inside strings; - enhanced numbers highlighting; - added optional highlighting for %-formatting inside strings; - added highlighting for some error conditions (wrong symbols in source file, mixing spaces and tabs, wrong number values, wrong %-formatting inside strings); - added highlighting for magic comments: source code encoding and #! (executable) strings; - added highlighting for new exceptions and builtins introduced in python 2.3, 2.4 and 2.5; - added highlighting for doctests; - added highlighting for new @decorator syntax introduced in Python 2.4a2; - added highlighting for trailing-space errors (triggered by new option: python_highlight_space_errors); - added highlighting for variable name errors; - added highlighting for hex number errors; -- http://mail.python.org/mailman/listinfo/python-list
Re: Middle matching - any Python library functions (besides re)?
Simon Forman wrote: Paul Rubin wrote: EP [EMAIL PROTECTED] writes: Given that I am looking for matches of all files against all other files (of similar length) is there a better bet than using re.search? The initial application concerns files in the 1,000's, and I could use a good solution for a number of files in the 100,000's. If these are text files, typically you'd use the Unix 'diff' utility to locate the differences. If you can, you definitely want to use diff. Otherwise, the difflib standard library module may be of use to you. Also, since you're talking about comparing many files to each other, you could pull out a substring of one file and use the 'in' operator to check if that substring is in another file. Something like this: f = open(filename) # or if binary open(filename, 'rb') f.seek(somewhere_in_the_file) substr = f.read(some_amount_of_data) f.close() try_diffing_us = [] for fn in list_of_filenames: data = open(fn).read() # or again open(fn, 'rb')... if substr in data: try_diffing_us.append(fn) # then diff just those filenames... That's a naive implementation but it should illustrate how to cut down on the number of actual diffs you'll need to perform. Of course, if your files are large it may not be feasible to do this with all of them. But they'd have to be really large, or there'd have to be lots and lots of them... :-) More information on your actual use case would be helpful in narrowing down the best options. Peace, ~Simon Would it be more efficient to checksum the files and then only diff the ones that fail a checksum compare? Utilizing the functions below may be of some help. #!/usr/bin/python # # # Function: generate and compare checksums on a file import md5, sys def getsum(filename): Generate the check sum based on received chunks of the file md5sum = md5.new() f = open(filename, 'r') for line in getblocks(f) : md5sum.update(line) f.close() return md5sum.hexdigest() def getblocks(f, blocksize=1024): Read file in small chunks to avoid having large files loaded into memory while True: s = f.read(blocksize) if not s: break yield s def checksum_compare(caller, cs='',check='', filename=''): Compare the generated and received checksum valued if cs != check: return 1 # compare failed else: return 0 # compare successful -- Adversity: That which does not kill me only postpones the inevitable. -- http://mail.python.org/mailman/listinfo/python-list
Re: Middle matching - any Python library functions (besides re)?
Simon Forman wrote: Andrew Robert wrote: Simon Forman wrote: Paul Rubin wrote: EP [EMAIL PROTECTED] writes: Given that I am looking for matches of all files against all other files (of similar length) is there a better bet than using re.search? The initial application concerns files in the 1,000's, and I could use a good solution for a number of files in the 100,000's. If these are text files, typically you'd use the Unix 'diff' utility to locate the differences. If you can, you definitely want to use diff. Otherwise, the difflib standard library module may be of use to you. Also, since you're talking about comparing many files to each other, you could pull out a substring of one file and use the 'in' operator to check if that substring is in another file. Something like this: f = open(filename) # or if binary open(filename, 'rb') f.seek(somewhere_in_the_file) substr = f.read(some_amount_of_data) f.close() try_diffing_us = [] for fn in list_of_filenames: data = open(fn).read() # or again open(fn, 'rb')... if substr in data: try_diffing_us.append(fn) # then diff just those filenames... That's a naive implementation but it should illustrate how to cut down on the number of actual diffs you'll need to perform. Of course, if your files are large it may not be feasible to do this with all of them. But they'd have to be really large, or there'd have to be lots and lots of them... :-) More information on your actual use case would be helpful in narrowing down the best options. Peace, ~Simon Would it be more efficient to checksum the files and then only diff the ones that fail a checksum compare? The thing about a checksum algorithm is that if you call it with some data, then change even one byte of the data and call it again, the resulting checksums will be (should be) very different. Checksumming the entire file contents won't help, but as bearophile said: If you can determine of a given a file where its heading garbage stops, then you can compute the signature just computing the python hash of some of the following bytes (30 or 200 byte may suffice). If the OP can do that then yes, it would likely be more efficient (although computing and comparing checksums on just 200 bytes or less might not be a significant gain on simply comparing the strings themselves.) But if he can't then the next best thing would be to take a subsection of the file somewhere after the heading cruft and search (using string 'in' string form) for that subsection in other files. (Actually there may be a better option than that, but I'm just not bright enough to have thought of it...) Utilizing the functions below may be of some help. #!/usr/bin/python # # # Function: generate and compare checksums on a file import md5, sys def getsum(filename): Generate the check sum based on received chunks of the file md5sum = md5.new() f = open(filename, 'r') for line in getblocks(f) : md5sum.update(line) f.close() return md5sum.hexdigest() def getblocks(f, blocksize=1024): Read file in small chunks to avoid having large files loaded into memory while True: s = f.read(blocksize) if not s: break yield s def checksum_compare(caller, cs='',check='', filename=''): Compare the generated and received checksum valued if cs != check: return 1 # compare failed else: return 0 # compare successful I'm curious why you included this elaborate function with it's unused args (caller and filename), unnecessary defaults, and it's odd inversion of Boolean values.. How is if not checksum_compare(something, sum0, sum1): #do something... any better than if sum0 == sum1: #do something... ? Peace, ~Simon Because I was lazy.. The checksume_compare came from something else I wrote that had special logging and e-mailer calls in it. Should have ripped the reference to caller and file name out.. -- http://mail.python.org/mailman/listinfo/python-list
Question regarding commit/backout of a message using the pymqi module
Hi everyone, Could someone help explain what I am doing wrong in this code block? This code block is an excerpt from a larger file that receives transmitted files via IBM WebSphere MQSeries an drops it to the local file system. Transmission of the file works as designed but it has a flaw. If the file cannot be created for whatever reason, the transmitted message is lost. What I am trying to do is ensure that a file transmit is considered successful only after the created file's checksum matches. If not, the code should treat it as an error and roll back the message to MQSeries without a commit. The basis for this should be around the pymqi.QueueManager class which is named mq in the block listed below. On execution, I get the traceback of: Traceback (most recent call last): File M:\MQ\MQ\Scripts\receiver.py, line 269, in ? receiver.run() File M:\MQ\MQ\Scripts\receiver.py, line 109, in run self.connect() File M:\MQ\MQ\Scripts\receiver.py, line 118, in connect self.qm.begin() File c:\python24\lib\site-packages\pymqi.py, line 738, in begin raise MQMIError(rv[0], rv[1]) pymqi.MQMIError: MQI Error. Comp: 1, Reason 2121: WARNING: MQRC_NO_EXTERNAL_PARTICIPANTS Do you have any idea why this might be occurring? class Receiver(object): def __init__(self,qm_name,queue_name): self.qm_name = qm_name self.queue_name = queue_name # Will be set later self.qm = None self.message = None def run(self): self.connect() self.get() def connect(self): Connect to queue manager try: self.qm = mq.QueueManager(options.qmanager.upper() ) self.qm.begin() except mq.PYIFError, err: mqevlog.event(error,err) sys.exit(1) def get(self): Get a message from queue. queue = mq.Queue(self.qm, self.queue_name) pmo = mq.pmo(Options = CMQC.MQPMO_SYNCPOINT) md = mq.md() while True: try: var = queue.get(self.message, md, pmo ) except mq.MQMIError,e: if e.reason != CMQC.MQRC_NO_MSG_AVAILABLE: mqevlog.event(error,e) sys.exit(1) break else: buff = StringIO(var) tree = ElementTree(file=buff) # Extract required elements and assign to local variables key = this should be a well-kept secret file_name = tree.find(dest).text creation_time = tree.find(creation_time).text contents = tree.find(contents).text check = tree.find(checksum).text #Decode temp file original = file_encoder.decode(contents) # Drop file to disk if os.path.exists(file_name) is False: open(file_name,wb).write(original) else: mqevlog.event(sys.argv[0],error,Output file path/name already exists) sys.exit(1) # Get checksum of newly created file sum=csums.getsum(file_name) # Compare checksum of created file with value transmitted if csums.checksum_compare(sys.argv[0],sum,check,file_name) == True: queue.commit() sys.exit(0) else: queue.backout() mqevlog.event(error,CheckSums of received/transmitted files do not match) sys.exit(1) Any help/insight you can provide on this would be greatly appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie Question
Saint Malo wrote: I am new to programming, and I've chosen python to start with. I wrote a simple program that asks several questions and assings each one of them a variable via raw_input command. I then combined all the variables into one like this a = b + c + d. After this I wrote these values to a file. What I want to do now is be able to search through the file for any data in there. Is this possible? Most certainly. It depends on how you want to treat the read data. for example # Open a file for read file=open(r'test.txt','r') # Read each line in file for line in file #Search each line if A in line: print I found A file.close() -- http://mail.python.org/mailman/listinfo/python-list
Re: __getattr__ question
If I remember correctly, this behavior depends on how the class is created (classic mode versus modern). Modern class foo(object): pass Classic ( pre python 2.2 I believe ) class foo(): pass The modern method of specifying object in the class definition gets all of the class attributes cleanly. The classic method does not have this so the behavior is simulated. Ben Finney wrote: Laszlo Nagy [EMAIL PROTECTED] writes: This is from the Python documentation (fragment): __getattr__( self, name) Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. This method should return the (computed) attribute value or raise an AttributeError exception. How can I determine if an attribute can be found in the usual places? When the attribute is not found in the usual places, the object's __getattr__ method (if it has one) is called. Thus, when you write your __getattr__ method, you should assume that the attribute has not been found in the usual places. -- http://mail.python.org/mailman/listinfo/python-list
Re: what are you using python language for?
I use python and the pymqi module to work with IBM WebSphere MQSeries and IBM WebSphere Message broker. -- http://mail.python.org/mailman/listinfo/python-list
regex/lambda black magic
Hi everyone, I have two test scripts, an encoder and a decoder. The encoder, listed below, works perfectly. import re,sys output = open(r'e:\pycode\out_test.txt','wb') for line in open(r'e:\pycode\sigh.txt','rb') : output.write( re.sub(r'([^\w\s])', lambda s: '%%%2X' % ord(s.group()), line)) The decoder, well, I have hopes. import re,sys output = open(r'e:\pycode\new_test.txt','wb') for line in open(r'e:\pycode\out_test.txt','rb') : output.write( re.sub(r'([^\w\s])', lambda s: chr(int(s.group(), 16)) % ord(s.group()), line)) The decoder generates the following traceback: Traceback (most recent call last): File E:\pycode\sample_decode_file_specials_from_hex.py, line 9, in ? output.write( re.sub(r'([^\w\s])', lambda s: chr(int(s.group(), 16)) % ord(s.group()), line)) File C:\Python24\lib\sre.py, line 142, in sub return _compile(pattern, 0).sub(repl, string, count) File E:\pycode\sample_decode_file_specials_from_hex.py, line 9, in lambda output.write( re.sub(r'([^\w\s])', lambda s: chr(int(s.group(), 16)) % ord(s.group()), line)) ValueError: invalid literal for int(): % Does anyone see what I am doing wrong? -- http://mail.python.org/mailman/listinfo/python-list
Re: regex/lambda black magic
Max Erickson wrote:
snip
/snip
Try getting rid of the lamba, it might make things clearer and it
simplifies debugging. Something like(this is just a sketch):
max
Yeah.. trying to keep everything on one line is becoming something of a
problem.
To make this easier, I followed something from another poster and came
up with this.
import re,base64
# Evaluate captured character as hex
def ret_hex(value):
return base64.b16encode(value)
def ret_ascii(value):
return base64.b16decode(value)
# Evaluate the value of whatever was matched
def eval_match(match):
return ret_ascii(match.group(0))
# Evaluate the value of whatever was matched
# def eval_match(match):
# return ret_hex(match.group(0))
out=open(r'e:\pycode\sigh.new2','wb')
# Read each line, pass any matches on line to function for
# line in file.readlines():
for line in open(r'e:\pycode\sigh.new','rb'):
print (re.sub('[^\w\s]',eval_match, line))
The char to hex pass works but omits the leading % at the start of each
hex value.
ie. 22 instead of %22
The hex to char pass does not appear to work at all.
No error is generated. It just appears to be ignored.
--
http://mail.python.org/mailman/listinfo/python-list
Re: regex/lambda black magic
Hi Everyone,
Thanks for all of your patience on this.
I finally got it to work.
Here is the completed test code showing what is going on.
Not cleaned up yet but it works for proof-of-concept purposes.
#!/usr/bin/python
import re,base64
# Evaluate captured character as hex
def ret_hex(value):
return '%'+base64.b16encode(value)
# Evaluate the value of whatever was matched
def enc_hex_match(match):
return ret_hex(match.group(0))
def ret_ascii(value):
return base64.b16decode(value)
# Evaluate the value of whatever was matched
def enc_ascii_match(match):
arg=match.group()
#remove the artifically inserted % sign
arg=arg[1:]
# decode the result
return ret_ascii(arg)
def file_encoder():
# Read each line, pass any matches on line to function for
# line in file.readlines():
output=open(r'e:\pycode\sigh.new','wb')
for line in open(r'e:\pycode\sigh.txt','rb'):
output.write( (re.sub('[^\w\s]',enc_hex_match, line)) )
output.close()
def file_decoder():
# Read each line, pass any matches on line to function for
# line in file.readlines():
output=open(r'e:\pycode\sigh.new2','wb')
for line in open(r'e:\pycode\sigh.new','rb'):
output.write(re.sub('%[0-9A-F][0-9A-F]',enc_ascii_match, line))
output.close()
file_encoder()
file_decoder()
--
http://mail.python.org/mailman/listinfo/python-list
Conversion of perl based regex to python method
I have two Perl expressions If windows: perl -ple s/([^\w\s])/sprintf(q#%%%2X#, ord $1)/ge somefile.txt If posix perl -ple 's/([^\w\s])/sprintf(%%%2X, ord $1)/ge' somefile.txt The [^\w\s] is a negated expression stating that any character a-zA-Z0-9_, space or tab is ignored. The () captures whatever matches and throws it into the $1 for processing by the sprintf In this case, %%%2X which is a three character hex value. How would you convert this to a python equivalent using the re or similar module? I've begun reading about using re expressions at http://www.amk.ca/python/howto/regex/ but I am still hazy on implementation. Any help you can provide would be greatly appreciated. Thanks, Andy -- http://mail.python.org/mailman/listinfo/python-list
Re: how to change sys.path?
Dennis Lee Bieber wrote: On Wed, 24 May 2006 14:45:55 GMT, John Salerno [EMAIL PROTECTED] declaimed the following in comp.lang.python: I just right-clicked on My Computer -- Properties -- Advanced -- Environment Variables, and added a new one called PYTHONPATH. I don't know if that edits the registry, but you don't *manually* have to edit the registry if you do it that way...unless of course you aren't supposed to be doing it that way! But it worked anyway. :) I may have gotten slightly confused -- I had an impression that, at least one poster in the thread, wanted to do this from within a Python program. That does go into registry modifications. For example, look at the entries under: (system environment, I believe) HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment (user specific environment) HKEY_CURRENT_USER\Environment Though in the case of PYTHONPATH, the core value seems to be in (for my install) HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.3\PythonPath which also has subkeys for Pythonwin, win32, and win32com Instead of messing with the registry, wouldn't it be easier to just add a line similar to this within your code? sys.path.append(r'\\mynetwork\share') I use something similar so that all my scripts can locate the same home-grown modules no matter where they are run from. -- http://mail.python.org/mailman/listinfo/python-list
Re: Conversion of perl based regex to python method
Andrew Robert wrote:
I have two Perl expressions
If windows:
perl -ple s/([^\w\s])/sprintf(q#%%%2X#, ord $1)/ge somefile.txt
If posix
perl -ple 's/([^\w\s])/sprintf(%%%2X, ord $1)/ge' somefile.txt
The [^\w\s] is a negated expression stating that any character
a-zA-Z0-9_, space or tab is ignored.
The () captures whatever matches and throws it into the $1 for
processing by the sprintf
In this case, %%%2X which is a three character hex value.
How would you convert this to a python equivalent using the re or
similar module?
I've begun reading about using re expressions at
http://www.amk.ca/python/howto/regex/ but I am still hazy on implementation.
Any help you can provide would be greatly appreciated.
Thanks,
Andy
Okay.. I got part of it..
The code/results below seem to do the first part of the expression.
I believe the next part is iterating across each of the characters,
evaluate the results and replace with hex as needed.
# Import the module
import re
# Open test file
file=open(r'm:\mq\mq\scripts\testme.txt','r')
# Read in a sample line
line=file.readline()
# Compile expression to exclude all characters plus space/tab
pattern=re.compile('[^\w\s]')
# Look to see if I can find a non-standard character
# from test line #! C:\Python24\Python
var=pattern.match('!')
# gotcha!
print var
_sre.SRE_Match object at 0x009DA8E0
# I got
print var.group()
!
# See if pattern will come back with something it shouldn't
var =pattern.match('C')
print var
#I got
None
Instead of being so linear, I was thinking that this might be closer.
Got to figure out the hex line but then we are golden
# Evaluate captured character as hex
def ret_hex(ch):
return chr((ord(ch) + 1) % )
# Evaluate the value of whatever was matched
def eval_match(match):
return ret_hex(match.group(0))
# open file
file = open(r'm:\mq\mq\scripts\testme.txt','r')
# Read each line, pass any matches on line to function
for line in file.readlines():
re.sub('[^\w\s]',eval_match, line)
--
http://mail.python.org/mailman/listinfo/python-list
Testing for file type
Hi Everyone, Is there a way to test if a file is binary or ascii within Python? I'd prefer not to text against file extension. Any help you can provide would be greatly appreciated. Thanks, Andy -- http://mail.python.org/mailman/listinfo/python-list
Class probkem - getting msg that self not defined
Hi Everyone, I am having a problem with a class and hope you can help. When I try to use the class listed below, I get the statement that self is not defined. test=TriggerMessage(data) var = test.decode(self.qname) I would have thought that self would have carried forward when I grabbed an instance of TriggerMessage. Any ideas on this? The class in question is: class TriggerMessage(object): def __init__(self,data): Unpacks the passed binary data based on the MQTCM2 format dictated in the MQ Application Programming Reference self.data=data self.structid=None self.version=None self.qname=None self.procname=None self.trigdata=None self.appltype=None self.applid=None self.envdata=None self.userdata=None self.qmgr=None def decode(self): import struct format='4s 4s 48s 48s 64s 4s 256s 128s 128s 48s' size=struct.calcsize(format) self.data=data self.structid, self.version, self.qname, self.processname, \ self.triggerdata, self.appltype, self.applid, \ self.envdata, self.userdata, self.qmgr \ = struct.unpack(format,self.data) -- http://mail.python.org/mailman/listinfo/python-list
Re: Class probkem - getting msg that self not defined
wes weston wrote: Andrew Robert wrote: Hi Everyone, I am having a problem with a class and hope you can help. When I try to use the class listed below, I get the statement that self is not defined. test=TriggerMessage(data) self is not known here; only inside the class. var = test.decode(self.qname) snip I guess I was barking up the wrong tree on that one. How would I go about getting the required values out of the class? Return self? -- http://mail.python.org/mailman/listinfo/python-list
File encoding strategy question
Hi everyone, I am in the process of creating a file transmit/receiver program using MQSeries. The way it works is through creation of an XML message. Elements within the XML message contain things such as file name, size, and the file contents. The file contents are encoded, currently using Base64, for support of both binary and text data. This works great but there is a small rub to the problem. I would like to be able to view the contents of the file if it is text while still maintaining the ability to transmit binary data. Does anyone know of an encoding format that would make this possible? The viewing of the file contents does not need to be perfect, merely enough that a particular in-transit file could be identified. Thoughts on this would be greatly appreciated. Thanks, Andy -- http://mail.python.org/mailman/listinfo/python-list
Re: 'error reading datastream' -- loading file only when transfer is complete?
[EMAIL PROTECTED] wrote: hello -- i'm running python/pygame on maemo (nokia 770). my situation is that i'm continually scouring this one directory for incoming files. if i see if there's a new file (coming in via wireless scp), i proceed to load it and process it. however, i think i am running into the issue that my program starts to load the file after it recognises there is new data, but before the file has completely transferred, so at unpredictable times i get a pygame.error: Error reading from datastream. what is the easiest way to work out this issue? easy being the key word. :) thank you very much! christine You might want to test for file locking before attempting to use -- http://mail.python.org/mailman/listinfo/python-list
Conversion of perl unpack code to python - something odd
Hey everyone,
Maybe you can see something I don't.
I need to convert a working piece of perl code to python.
The perl code is:
sub ParseTrig {
# unpack the ibm struct that triggers messages.
my $struct = shift;
my %data;
@data{qw/StructID Version QName ProcessName TriggerData ApplType
ApplId EnvData UserData QMgrName/} =
unpack(A4A4A48A48A64A4A256A128A128A48, $struct);
return undef unless $data{StructID} eq 'TMC';
return \%data;
Taking away the fact that it is a function, the base code sets a 732
element structure with the ascii pattern derived above.
I wrote a simple test script that accepts the command argument at
position 1.
#!C:\Python24\python
import sys, string, struct
# From the language declarations in manual, the following format was derived
format='4s 4s 48s 48s 64s 4s 256s 128s 128s 48s'
size=struct.calcsize(format)
# Extract list item out for convenience/readability
data=sys.argv[1]
# Calculated size of the struct format is 732
# Length of data string is 732
print len(data)
print size
d1,d2=struct.unpack(format,data)
print d1
sys.exit(0)
When I run it, it says there are too many values to unpack.
Traceback ( most recent call last)
File m:\mq\mq\scripts\format.py, line 32, in ?
d1, d2 = struct.unpack(format,data)
ValueError: too many values to unpack
Error starting triggered application
I checked the manual on the structure used to pack the code and it
appears correct.
#
# MQTMC2 Language declarations as defined chapt 22 of programmers ref
# http://publibfp.boulder.ibm.com/epubs/pdf/csqzak09.pdf
#
# CHAR4 Struct ID
# CHAR4 Version
# CHAR48 QName
# CHAR48 ProcessName
# CHAR64 TriggerData
# CHAR4 ApplType
# CHAR256 ApplID
# CHAR128 EnvData
# CHAR128 UserData
# CHAR48 QMgrName
Any help you can provide on this would be greatly appreciated.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Conversion of perl unpack code to python - something odd
Peter Otten wrote: Andrew Robert wrote: format='4s 4s 48s 48s 64s 4s 256s 128s 128s 48s' You are trying to squeeze 10 items into just d1,d2=struct.unpack(format,data) two variables (d1 and d2) ValueError: too many values to unpack and Python is quite explicit that it doesn't like that once you realize that 'unpack' doesn't refer to struct.unpack() but to tuple unpacking like a, b = ab a, b = abc Traceback (most recent call last): File stdin, line 1, in ? ValueError: too many values to unpack Peter Now I feel like a first class idiot. Thanks for the help. Added the extra eight variables and things worked perfectly. Going to go stand in the corner for an hour now :) -- http://mail.python.org/mailman/listinfo/python-list
Re: number of different lines in a file
r.e.s. wrote: I have a million-line text file with 100 characters per line, and simply need to determine how many of the lines are distinct. On my PC, this little program just goes to never-never land: def number_distinct(fn): f = file(fn) x = f.readline().strip() L = [] while x'': if x not in L: L = L + [x] x = f.readline().strip() return len(L) Would anyone care to point out improvements? Is there a better algorithm for doing this? Take a look at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52560 It is a python approach to the uniq command on *nix. -- http://mail.python.org/mailman/listinfo/python-list
Re: Option parser question - reading options from file as well as command line
Tim N. van der Leeuw wrote:
Andrew Robert wrote:
Hi Everyone.
I tried the following to get input into optionparser from either a file
or command line.
The code below detects the passed file argument and prints the file
contents but the individual swithces do not get passed to option parser.
After reading your post I decided to play around with optparse a bit,
to get acquainted with it.
Experimenting with IDLE I found that the Values object returned by
parse_args has a method 'readfile', and this 'readfile' method allows
you to add options to the Values object.
The syntax should be in the form:
option=value
option should not include the hyphens -- if your option is added as
'-x', then you should write:
x=3
not:
-x=3
If you have options with both long names and short names, then you
should use the long name -- if your option is added as '-f', '--file',
then you should write:
file=foo.txt
not:
f=foo.txt
Also, you need the assignment-operator in your file.
I didn't find any documentation on this, and I didn't try this with any
actions; only with options added to the parser like
op.add_option('-x', dest='x')
Using this 2-step approach allows you to use optparse itself to get to
the command-line option with your command-line settings...
Out of pure personal interest, what queuing-system are you writing to
from Python? What libraries do you have for that? And is it commercial
software, or freely downloadable?
(I'd love to be able to write messages to IBM MQ Series from Python)
Cheers,
--Tim
Hi Tim,
I am using the pymqi module which is freely available at
http://pymqi.sourceforge.net/ .
Documentation on the module can be found at
http://pymqi.sourceforge.net/pymqidoc.html .
I have a few python examples on my web site located at
http://home.townisp.com/~arobert/
There are also a lot of good examples at
http://www.koders.com/info.aspx?c=ProjectInfopid=TVM5FGBZMY4E5ZH7GC9AX54PAC
.
If you come up with anything, I would be glad to see what you have.
Back to the original issue:
I'm not sure exactly what you mean about the readfile option and format.
Could you send me a code snippet so I can get a better feel for it?
Thanks,
Andy
--
http://mail.python.org/mailman/listinfo/python-list
Process forking on Windows
Hi everyone, I have a python program that will need to interact with an MQSeries trigger monitor. It does this fine but it hogs the trigger monitor while it executes. I'd like to fork the program off and terminate the parent process so that the trigger monitor frees up. Does anyone how this can be accomplished on a Windows platform? I've looked at doing this via the subprocess module but this doesn't look like the correct approach. Any help you can provide would be greatly appreciated. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Process forking on Windows
bruno at modulix wrote: Andrew Robert wrote: Hi everyone, I have a python program that will need to interact with an MQSeries trigger monitor. It does this fine but it hogs the trigger monitor while it executes. I'd like to fork the program off and terminate the parent process so that the trigger monitor frees up. just-asking Is this really the solution ? /just-asking Does anyone how this can be accomplished on a Windows platform? AFAIK, there's no fork in Windows - you might want to give a try with CreateProcess. http://www.byte.com/art/9410/sec14/art3.htm http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnucmg/html/UCMGch01.asp HTH Unfortunately there is a real need for this. The MQSeries trigger monitor is single threaded. Because of this, my program would absorb it until it completes. The way to get around this would be to fork off and terminate the parent. Unfortunately, Windows appears to be somewhat stubborn about it. Creating a subprocess does not alleviate the need to get the originating process out of the trigger monitor. -- http://mail.python.org/mailman/listinfo/python-list
Re: Process forking on Windows - or what is MQSeries
Gary Herron wrote: Andrew Robert wrote: snip The windows CreateProcess call has many of the same semantics as the Unix fork, i.e., a new process is created sharing all the resources of the original process. The subprocess modules uses CreateProcess, but if that does not give you sufficient control over the process creation, you can call CreateProcess directly via the win32process module in the win32all package. However, I still don't understand *what* the MQSeries trigger monitor is or *how* it would create the need for such a solution. Gary Herron MQSeries is a rather interesting piece of middle-ware offered by IBM that allows you to link disparate hosts/applications together via XML messaging and application specific queues. In the broadest sense, think of MQSeries like a large switchboard connecting everything together. Message queues can be programmed to do something via a mq application process such as a rule to take action when first, 5th, etc message arrives. The state of queues and their linked processes are controlled by the trigger monitor. The trigger monitor can only deal with one running process at a time. In this situation, it is possible for a process(my python program) to monopolize and block other processes from being triggered. Ideally, this needs to be avoided through the use of a fork. If you are interested, you can get all kinds of Websphere MQSeries and Websphere Message Broker information at http://www-306.ibm.com/software/integration/wmq/library/ On a side note, I use the pymqi module to make calls to MQSeries. -- http://mail.python.org/mailman/listinfo/python-list
Option parser question - reading options from file as well as command line
Hi Everyone. I tried the following to get input into optionparser from either a file or command line. The code below detects the passed file argument and prints the file contents but the individual swithces do not get passed to option parser. Doing a test print of options.qmanager shows it unassigned. Any ideas? # # Parse command line options and automatically build help/usage # display # if len(sys.argv) == 2: infile= open(sys.argv[1],rb).read() print infile parser=OptionParser( infile ) else: parser = OptionParser() parser.add_option(-m,--qmanager, dest=qmanager, help=\t\tQueue Manager to inquire against), parser.add_option(-q,--queue, dest=queue, help=\t\tQueue the message will be sent to), parser.add_option(-t,--to, dest=mto, help=\t\taddress any mail messages will be sent to) (options, args) = parser.parse_args() print options.qmanager -- http://mail.python.org/mailman/listinfo/python-list
Re: Option parser question - reading options from file as well as command line
Max Erickson wrote: Andrew Robert [EMAIL PROTECTED] wrote in news:[EMAIL PROTECTED]: snip \snip Check parser.usage, it is likely to look a lot like your infile. I'm not sure, but I think you need to pass your alternative arguments to parser.parse_args. max Hi Max, I tried passing in the read line like so: infile= open(sys.argv[1],rb).read() parser=OptionParser() print infile (options, args) = parser.parse_args(infile) I end up getting the following traceback. Traceback (most recent call last): File C:\Documents and Settings\Andrew Robert\My Documents\receiver.py, line 327, in ? (options, args) = parser.parse_args(infile) File C:\Python24\lib\optparse.py, line 1275, in parse_args stop = self._process_args(largs, rargs, values) File C:\Python24\lib\optparse.py, line 1322, in _process_args del rargs[0] TypeError: object doesn't support item deletion Any ideas? -- http://mail.python.org/mailman/listinfo/python-list
Re: Option parser question - reading options from file as well as command line
Max Erickson wrote:
I don't know much about optparse, but since I was bored:
help(o.parse_args)
Help on method parse_args in module optparse:
parse_args(self, args=None, values=None) method of
optparse.OptionParser instance
parse_args(args : [string] = sys.argv[1:],
values : Values = None)
- (values : Values, args : [string])
Parse the command-line options found in 'args' (default:
sys.argv[1:]). Any errors result in a call to 'error()', which
by default prints the usage message to stderr and calls
sys.exit() with an error message. On success returns a pair
(values, args) where 'values' is an Values instance (with all
your option values) and 'args' is the list of arguments left
over after parsing options.
o.parse_args('seven')
Traceback (most recent call last):
File pyshell#15, line 1, in ?
o.parse_args('seven')
File C:\bin\Python24\lib\optparse.py, line 1275, in parse_args
stop = self._process_args(largs, rargs, values)
File C:\bin\Python24\lib\optparse.py, line 1322, in _process_args
del rargs[0]
TypeError: object doesn't support item deletion
That's the result of poking an optionParser instance in Idle.
parse_args is expecting something that looks like sys.argv[1:], which
is a list. You are passing it a string.
max
Yup.. the code now works as:
parser = OptionParser()
if len(sys.argv) == 2:
lines = open(sys.argv[1],rb).readlines()
for line in lines:
line=line.strip()
if not line:
continue
short, long, dest, help, default = line.split(;)
help = \t\t + help # Prepend tabs to help message
parser.add_option(short, long, dest=dest, help=help,
default=default)
else:
parser.add_option(-m,--qmanager, dest=qmanager,
help=\t\tQueue Manager to inquire against),
parser.add_option(-q,--queue, dest=queue,
help=\t\tQueue the message will be sent to),
parser.add_option(-d,--dest, dest=dest,
help=\t\tDestination File Name),
parser.add_option(-f, --file,
action=store, type=string, dest=filename,
help=File to be transmitted, metavar=FILE)
(options, args) = parser.parse_args()
thanks all for the insight
--
http://mail.python.org/mailman/listinfo/python-list
Re: Question regarding checksuming of a file
When I run the script, I get an error that the file object does not have the attribute getblocks. Did you mean this instead? def getblocks(f, blocksize=1024): while True: s = f.read(blocksize) if not s: return yield s def getsum(self): md5sum = md5.new() f = open(self.file_name, 'rb') for line in getblocks(f) : md5sum.update(line) f.close() return md5sum.hexdigest() -- http://mail.python.org/mailman/listinfo/python-list
Question regarding checksuming of a file
Good evening, I need to generate checksums of a file, store the value in a variable, and pass it along for later comparison. The MD5 module would seem to do the trick but I'm sketchy on implementation. The nearest I can see would be import md5 m=md5.new() contents = open(self.file_name,rb).read() check=md5.update(contents) However this does not appear to be actually returning the checksum. Does anyone have insight into where I am going wrong? Any help you can provide would be greatly appreciated. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Question regarding checksuming of a file
Actually, I think I got it but would like to confirm this looks right. import md5 checksum = md5.new() mfn = open(self.file_name, 'r') for line in mfn.readlines(): checksum.update(line) mfn.close() cs = checksum.hexdigest() print cs The value cs should contain the MD5 checksum or did I miss something? Any help you can provide would be greatly appreciated. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Question regarding checksuming of a file
Roy Smith wrote: However this does not appear to be actually returning the checksum. Does anyone have insight into where I am going wrong? After calling update(), you need to call digest(). Update() only updates the internal state of the md5 state machine; digest() returns the hash. Also, for the code above, it's m.update(), not md5.update(). Update() is a method of an md5 instance object, not the md5 module itself. Lastly, the md5 algorithm is known to be weak. If you're doing md5 to maintain compatability with some pre-existing implementation, that's one thing. But, if you're starting something new from scratch, I would suggest using SHA-1 instead (see the sha module). SHA-1 is much stronger cryptographically than md5. The Python API is virtually identical, so it's no added work to switch to the stronger algorithm. Hi Roy, This is strictly for checking if a file was corrupted during transit over an MQSeries channel. The check is not intended to be used for crypto purposes. -- http://mail.python.org/mailman/listinfo/python-list
MQSeries based file transfers using the pymqi module
Hi everyone, Has anyone developed a pymqi module based file transfer method for use with WebSphere MQSeries v5.3? If so, would it be possible to point me towards examples of how this was done? Any help that can be provided would be greatly appreciated. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Using Python to interact with BMC Patrol
Hi Everyone, Has anyone done any Python coding to manage/interact/customize BMC Patrol? If anyone has, could you please point me to where I can find documentation/guides on this? I checked the Python SIGs and Vault of Parnasus but didn't see anything available. Any insight you might have on this would be greatly appreciated. Thank you, Andy -- http://mail.python.org/mailman/listinfo/python-list
Re: screen clear question
Nick Coghlan wrote: Alan Gauld wrote: But the bottom line is that there is no builtin command because the mechanism is different on each platform. I'd have said it was because the inpreter is line-oriented rather than screen-oriented, but YMMV. Cheers, Nick. I would try doing a test against the resident OS the program is running against and set the clear command based on that. -- Thank you, Andrew Robert E-mail: [EMAIL PROTECTED] Ur: http://shardservant.no-ip.info -- http://mail.python.org/mailman/listinfo/python-list
Re: How to make executable file ?
BOOGIEMAN wrote: Just how to make *.exe file from python code ?? I typed this : a, b = 0, 1 while b 1000: print b, a, b = b, a+b and saved it as pyt.txt Now, how do I make pyt.exe file ??? I want to run it on windows where isn't installed python. You may want to try cx_freeze. Details on it can be found at http://starship.python.net/crew/atuining/cx_Freeze/ Binaries are available for Linux and Windows. Alternately, source code is available if you need to compile it for a different platform. -- Thank you, Andrew Robert E-mail: [EMAIL PROTECTED] Ur: http://shardservant.no-ip.info -- http://mail.python.org/mailman/listinfo/python-list
Re: Python mascot proposal
What about a dead camel? -- http://mail.python.org/mailman/listinfo/python-list
