Re: Comparison of parsers in python?

2009-09-26 Thread andrew cooke
On Sep 21, 10:59 am, Nobody nob...@nowhere.com wrote: I have a similar question. What I want: a tokeniser generator which can take a lex-style grammar (not necessarily lex syntax, but a set of token specifications defined by REs, BNF, or whatever), generate a DFA, then run the DFA on

Re: Comparison of parsers in python?

2009-09-25 Thread Nobody
On Tue, 22 Sep 2009 17:07:33 +1200, greg wrote: What I want: a tokeniser generator which can take a lex-style grammar (not necessarily lex syntax, but a set of token specifications defined by REs, BNF, or whatever), generate a DFA, then run the DFA on sequences of bytes. It must allow the

Re: Comparison of parsers in python?

2009-09-21 Thread Nobody
On Sat, 19 Sep 2009 13:21:58 -0700, Peng Yu wrote: I did a google search and found various parser in python that can be used to parse different files in various situation. I don't see a page that summarizes and compares all the available parsers in python, from simple and easy-to-use ones to

Re: Comparison of parsers in python?

2009-09-21 Thread greg
Nobody wrote: What I want: a tokeniser generator which can take a lex-style grammar (not necessarily lex syntax, but a set of token specifications defined by REs, BNF, or whatever), generate a DFA, then run the DFA on sequences of bytes. It must allow the syntax to be defined at run-time. You

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 19, 9:34 pm, Peng Yu pengyu...@gmail.com wrote: On Sep 19, 6:05 pm, Robert Kern robert.k...@gmail.com wrote: http://nedbatchelder.com/text/python-parsers.html This is more a less just a list of parsers. I would like some detailed guidelines on which one to choose for various parsing

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 6:50 AM, andrew cooke and...@acooke.org wrote: On Sep 19, 9:34 pm, Peng Yu pengyu...@gmail.com wrote: On Sep 19, 6:05 pm, Robert Kern robert.k...@gmail.com wrote: http://nedbatchelder.com/text/python-parsers.html This is more a less just a list of parsers. I would like

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 20, 8:11 am, Peng Yu pengyu...@gmail.com wrote: On Sun, Sep 20, 2009 at 6:50 AM, andrew cooke and...@acooke.org wrote: On Sep 19, 9:34 pm, Peng Yu pengyu...@gmail.com wrote: On Sep 19, 6:05 pm, Robert Kern robert.k...@gmail.com wrote: http://nedbatchelder.com/text/python-parsers.html

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 19, 11:39 pm, TerryP bigboss1...@gmail.com wrote: [...] For flat data, simple unix style rc or dos style ini file will often suffice, and writing a parser is fairly trivial; in fact writing a [...] python already includes parsers for .ini configuration files. [...] The best way to

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
One word of warning - the documentation for that format says at the beginning that it is compressed in some way. I am not sure if that means within some program, or on disk. But most parsers will not be much use with a compressed file - you will need to uncompress it first. --

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 7:20 AM, andrew cooke and...@acooke.org wrote: On Sep 20, 8:11 am, Peng Yu pengyu...@gmail.com wrote: On Sun, Sep 20, 2009 at 6:50 AM, andrew cooke and...@acooke.org wrote: On Sep 19, 9:34 pm, Peng Yu pengyu...@gmail.com wrote: On Sep 19, 6:05 pm, Robert Kern

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
The file size of a wig file can be very large (GB). Most tasks on this file format does not need the parser to save all the lines read from the file in the memory to produce the parsing result. I'm wondering if pyparsing is capable of parsing large wig files by keeping only minimum required

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
also, parsing large files may be slow. in which case you may be better with a non-python solution (even if you call it from python). your file format is so simple that you may find a lexer is enough for what you want, and they should be stream oriented. have a look at the shlex package that is

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 8:19 AM, andrew cooke and...@acooke.org wrote: also, parsing large files may be slow.  in which case you may be better with a non-python solution (even if you call it from python). your file format is so simple that you may find a lexer is enough for what you want,

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
I don't quite understand this point.  If I don't use a parser, since python can read numbers line by line, why I need a lexer package? for the lines of numbers it would make no difference; for the track definition lines it would save you some work. as you said, this is a simple format, so the

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 8:49 AM, andrew cooke and...@acooke.org wrote: I don't quite understand this point.  If I don't use a parser, since python can read numbers line by line, why I need a lexer package? for the lines of numbers it would make no difference; for the track definition lines it

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
So for the track definition, using a lexer package would be better than using regex in python, right? they are similar. a lexer is really just a library that packages regular expressions in a certain way. so you could write your own code and you would really be writing a simple lexer. the

Re: Comparison of parsers in python?

2009-09-20 Thread Robert Kern
Peng Yu wrote: The file size of a wig file can be very large (GB). Most tasks on this file format does not need the parser to save all the lines read from the file in the memory to produce the parsing result. I'm wondering if pyparsing is capable of parsing large wig files by keeping only

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 20, 9:12 am, andrew cooke and...@acooke.org wrote: ps is there somewhere can download example files?  this would be useful for my own testing.  thanks. i replied to a lot of your questions here; any chance you could reply to this one of mine? the wig format looks like it could be a good

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 1:35 PM, andrew cooke and...@acooke.org wrote: On Sep 20, 9:12 am, andrew cooke and...@acooke.org wrote: ps is there somewhere can download example files?  this would be useful for my own testing.  thanks. i replied to a lot of your questions here; any chance you could

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 20, 3:16 pm, Peng Yu pengyu...@gmail.com wrote: On Sun, Sep 20, 2009 at 1:35 PM, andrew cooke and...@acooke.org wrote: On Sep 20, 9:12 am, andrew cooke and...@acooke.org wrote: ps is there somewhere can download example files?  this would be useful for my own testing.  thanks. i

Re: Comparison of parsers in python?

2009-09-20 Thread Simon Forman
On Sep 19, 9:34 pm, Peng Yu pengyu...@gmail.com wrote: On Sep 19, 6:05 pm, Robert Kern robert.k...@gmail.com wrote: Peng Yu wrote: Hi, I did a google search and found various parser in python that can be used to parse different files in various situation. I don't see a page that

Comparison of parsers in python?

2009-09-19 Thread Peng Yu
Hi, I did a google search and found various parser in python that can be used to parse different files in various situation. I don't see a page that summarizes and compares all the available parsers in python, from simple and easy-to-use ones to complex and powerful ones. I am wondering if

Re: Comparison of parsers in python?

2009-09-19 Thread Robert Kern
Peng Yu wrote: Hi, I did a google search and found various parser in python that can be used to parse different files in various situation. I don't see a page that summarizes and compares all the available parsers in python, from simple and easy-to-use ones to complex and powerful ones.

Re: Comparison of parsers in python?

2009-09-19 Thread Peng Yu
On Sep 19, 6:05 pm, Robert Kern robert.k...@gmail.com wrote: Peng Yu wrote: Hi, I did a google search and found various parser in python that can be used to parse different files in various situation. I don't see a page that summarizes and compares all the available parsers in python,

Re: Comparison of parsers in python?

2009-09-19 Thread TerryP
Peng Yu wrote: This is more a less just a list of parsers. I would like some detailed guidelines on which one to choose for various parsing problems. Regards, Peng It depends on the parsing problem. Obviously your not going to use an INI parser to work with XML, or vice versa. Likewise