Re: [Tutor] reading output from a c executable.

2008-12-22 Thread Ravi Kondamuru
I tried what Bill Campbell suggested: read the len first and then use that to populate the structdef length field for the string len = xstruct.structdef(xstruct.little_endian, [ ('len', (xstruct. unsigned_long, 1)), ]) l = len(buf[0:3]) rec = xstruct.structdef(xstruct.little_endian, [

Re: [Tutor] reading output from a c executable.

2008-12-20 Thread Ravi Kondamuru
I am trying to use xstruct module to unpack a varaible size record with the following structure. struct nameid { u32bits len /* total length */ u32bits id; char name; /* name variable length */ } As can be seen the length of the name = len - (sizeof(len) + sizeof(id)). How do I use

Re: [Tutor] reading output from a c executable.

2008-12-20 Thread Bill Campbell
On Sat, Dec 20, 2008, Ravi Kondamuru wrote: I am trying to use xstruct module to unpack a varaible size record with the following structure. struct nameid { u32bits len /* total length */ u32bits id; char name; /* name variable length */ } As can be seen the length of the name = len -

Re: [Tutor] reading output from a c executable.

2008-12-20 Thread Alan Gauld
Ravi Kondamuru ravikondam...@gmail.com wrote struct nameid { u32bits len /* total length */ u32bits id; char name; /* name variable length */ } As can be seen the length of the name = len - (sizeof(len) + sizeof(id)). How do I use xstruct or struct to unpack such a structure? I

Re: [Tutor] reading output from a c executable.

2008-12-20 Thread Alan Gauld
Bill Campbell b...@celestial.net wrote struct nameid { u32bits len /* total length */ u32bits id; char name; /* name variable length */ } I haven't done this in python yet, but when handling things like this in C and perl, I have done it with two reads, the first to get the length,

Re: [Tutor] reading output from a c executable.

2008-12-12 Thread Lie Ryan
On Fri, 12 Dec 2008 03:13:16 -0800, Ravi Kondamuru wrote: Denis, These are 32bit, 64bit counters (essentially numbers). Bob, There are well over 10K counters in the log file that are updated every 5 secs. If a counter1's graph was requested, log will have to be parsed once to get the data

Re: [Tutor] reading output from a c executable.

2008-12-12 Thread Ravi Kondamuru
Denis, These are 32bit, 64bit counters (essentially numbers). Bob, There are well over 10K counters in the log file that are updated every 5 secs. If a counter1's graph was requested, log will have to be parsed once to get the data points. If a user asked for a counter2, now it needs to be

Re: [Tutor] reading output from a c executable.

2008-12-12 Thread bob gailer
Ravi Kondamuru wrote: Denis, These are 32bit, 64bit counters (essentially numbers). Bob, There are well over 10K counters in the log file that are updated every 5 secs. If a counter1's graph was requested, log will have to be parsed once to get the data points. If a user asked for a counter2,

Re: [Tutor] reading output from a c executable.

2008-12-12 Thread bob gailer
Ravi Kondamuru wrote: Denis, These are 32bit, 64bit counters (essentially numbers). Bob, There are well over 10K counters in the log file that are updated every 5 secs. If a counter1's graph was requested, log will have to be parsed once to get the data points. If a user asked for a counter2,

[Tutor] reading output from a c executable.

2008-12-11 Thread Ravi Kondamuru
Hi, I am writing a script to read list output from a C executable. How should c program be written so that python can read the output as a list? Any pointers to info on this appreciated. thanks, Ravi. ___ Tutor maillist - Tutor@python.org

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread bob gailer
Ravi Kondamuru wrote: Hi, I am writing a script to read list output from a C executable. How should c program be written so that python can read the output as a list? Any pointers to info on this appreciated. Funny that a C programmer is asking for pointers, when C has lots of pointers and

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread Steve Willoughby
Ravi Kondamuru wrote: Hi, I am writing a script to read list output from a C executable. How should c program be written so that python can read the output as a list? Any pointers to info on this appreciated. The possibilities are truly wide open on this. Python can read a variety of

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread Ravi Kondamuru
I am expecting these lists to be huge and was hoping to avoid re-parsing in python. Any way for the c program to return a list that python can directly use. Thanks for the pointer to json :) I am going to explore and evaluate re-parsing overhead. thanks, Ravi. On Thu, Dec 11, 2008 at 10:19 AM,

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread Ravi Kondamuru
I am trying to read a binary log file to extract system counters. These counters will then be used to generate web-based graphs using the chart-director api in python. For extracting the counters I planning to write a program in C to read and give the output as lists for use by chart-director. If

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread bob gailer
Ravi Kondamuru wrote: I am expecting these lists to be huge and was hoping to avoid re-parsing in python. Any way for the c program to return a list that python can directly use. Not as far as I know. Lists in Python are internal objects. If you were able to package the C and Python programs

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread bob gailer
Ravi Kondamuru wrote: I am trying to read a binary log file to extract system counters. These counters will then be used to generate web-based graphs using the chart-director api in python. For extracting the counters I planning to write a program in C to read and give the output as lists for

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread Steve Willoughby
Ravi Kondamuru wrote: I am expecting these lists to be huge and was hoping to avoid re-parsing in python. Any way for the c program to return a list that python can Mind if I ask the obvious question here? Why are you wanting to avoid parsing in Python? Any time you have one program (in

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread Steve Willoughby
Ravi Kondamuru wrote: I am trying to read a binary log file to extract system counters. These counters will then be used to generate web-based graphs using the chart-director api in python. For extracting the counters I planning to write a program in C to read and give the output as lists for

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread Ravi Kondamuru
This was a great info exchange for me. I am going to try out the all-python solution and see how it works out. thank you all, Ravi. On Thu, Dec 11, 2008 at 10:59 AM, Ravi Kondamuru ravikondam...@gmail.comwrote: Reasons for C: 1. The log files I am working are 60-100MB files. I *assumed* using

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread Eike Welk
On Thursday 11 December 2008, Ravi Kondamuru wrote: This was a great info exchange for me. I am going to try out the all-python solution and see how it works out. thank you all, You could try to optimize slow parts with the compiled Cython language. Nearly all Python code is legal Cython; but

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread spir
Ravi Kondamuru a écrit : I am trying to read a binary log file to extract system counters. These counters will then be used to generate web-based graphs using the chart-director api in python. For extracting the counters I planning to write a program in C to read and give the output as lists for

Re: [Tutor] reading output from a c executable.

2008-12-11 Thread bob gailer
Ravi Kondamuru wrote: Reasons for C: 1. The log files I am working are 60-100MB files. I *assumed* using C will reduce the parse time. 2. The records themselves are variable length and hence was concerned about the complexity for implementation in python. 3. Since I am not using a database,