Re: Comparison of parsers in python?

2009-09-26 Thread andrew cooke
On Sep 21, 10:59 am, Nobody wrote: > I have a similar question. > > What I want: a tokeniser generator which can take a lex-style grammar (not > necessarily lex syntax, but a set of token specifications defined by > REs, BNF, or whatever), generate a DFA, then run the DFA on sequences of > bytes.

Re: Comparison of parsers in python?

2009-09-25 Thread Nobody
On Tue, 22 Sep 2009 17:07:33 +1200, greg wrote: >> What I want: a tokeniser generator which can take a lex-style grammar (not >> necessarily lex syntax, but a set of token specifications defined by >> REs, BNF, or whatever), generate a DFA, then run the DFA on sequences of >> bytes. It must allow

Re: Comparison of parsers in python?

2009-09-21 Thread greg
Nobody wrote: What I want: a tokeniser generator which can take a lex-style grammar (not necessarily lex syntax, but a set of token specifications defined by REs, BNF, or whatever), generate a DFA, then run the DFA on sequences of bytes. It must allow the syntax to be defined at run-time. You

Re: Comparison of parsers in python?

2009-09-21 Thread Nobody
On Sat, 19 Sep 2009 13:21:58 -0700, Peng Yu wrote: > I did a google search and found various parser in python that can be > used to parse different files in various situation. I don't see a page > that summarizes and compares all the available parsers in python, from > simple and easy-to-use ones

Re: Comparison of parsers in python?

2009-09-20 Thread Simon Forman
On Sep 19, 9:34 pm, Peng Yu wrote: > On Sep 19, 6:05 pm, Robert Kern wrote: > > > Peng Yu wrote: > > > Hi, > > > > I did a google search and found various parser in python that can be > > > used to parse different files in various situation. I don't see a page > > > that summarizes and compares a

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 20, 3:16 pm, Peng Yu wrote: > On Sun, Sep 20, 2009 at 1:35 PM, andrew cooke wrote: > > On Sep 20, 9:12 am, andrew cooke wrote: > >> ps is there somewhere can download example files?  this would be > >> useful for my own testing.  thanks. > > > i replied to a lot of your questions here; an

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 1:35 PM, andrew cooke wrote: > On Sep 20, 9:12 am, andrew cooke wrote: >> ps is there somewhere can download example files?  this would be >> useful for my own testing.  thanks. > > i replied to a lot of your questions here; any chance you could reply > to this one of mine

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 20, 9:12 am, andrew cooke wrote: > ps is there somewhere can download example files?  this would be > useful for my own testing.  thanks. i replied to a lot of your questions here; any chance you could reply to this one of mine? the wig format looks like it could be a good test for lepl.

Re: Comparison of parsers in python?

2009-09-20 Thread Robert Kern
Peng Yu wrote: The file size of a wig file can be very large (GB). Most tasks on this file format does not need the parser to save all the lines read from the file in the memory to produce the parsing result. I'm wondering if pyparsing is capable of parsing large wig files by keeping only minimu

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
> So for the track definition, using a lexer package would be better > than using regex in python, right? they are similar. a lexer is really just a library that packages regular expressions in a certain way. so you could write your own code and you would really be writing a simple lexer. the a

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 8:49 AM, andrew cooke wrote: >> I don't quite understand this point.  If I don't use a parser, since >> python can read numbers line by line, why I need a lexer package? > > for the lines of numbers it would make no difference; for the track > definition lines it would save

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
> I don't quite understand this point.  If I don't use a parser, since > python can read numbers line by line, why I need a lexer package? for the lines of numbers it would make no difference; for the track definition lines it would save you some work. as you said, this is a simple format, so the

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 8:19 AM, andrew cooke wrote: > > also, parsing large files may be slow.  in which case you may be > better with a non-python solution (even if you call it from python). > > your file format is so simple that you may find a lexer is enough for > what you want, and they shoul

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
also, parsing large files may be slow. in which case you may be better with a non-python solution (even if you call it from python). your file format is so simple that you may find a lexer is enough for what you want, and they should be stream oriented. have a look at the "shlex" package that i

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
> The file size of a wig file can be very large (GB). Most tasks on this > file format does not need the parser to save all the lines read from > the file in the memory to produce the parsing result. I'm wondering if > pyparsing is capable of parsing large wig files by keeping only > minimum requir

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 7:20 AM, andrew cooke wrote: > On Sep 20, 8:11 am, Peng Yu wrote: >> On Sun, Sep 20, 2009 at 6:50 AM, andrew cooke wrote: >> > On Sep 19, 9:34 pm, Peng Yu wrote: >> >> On Sep 19, 6:05 pm, Robert Kern wrote: >> >> >http://nedbatchelder.com/text/python-parsers.html >> >>

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
One word of warning - the documentation for that format says at the beginning that it is compressed in some way. I am not sure if that means within some program, or on disk. But most parsers will not be much use with a compressed file - you will need to uncompress it first. -- http://mail.pytho

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 20, 8:11 am, Peng Yu wrote: > On Sun, Sep 20, 2009 at 6:50 AM, andrew cooke wrote: > > On Sep 19, 9:34 pm, Peng Yu wrote: > >> On Sep 19, 6:05 pm, Robert Kern wrote: > >> >http://nedbatchelder.com/text/python-parsers.html > > >> This is more a less just a list of parsers. I would like so

Re: Comparison of parsers in python?

2009-09-20 Thread Peng Yu
On Sun, Sep 20, 2009 at 6:50 AM, andrew cooke wrote: > On Sep 19, 9:34 pm, Peng Yu wrote: >> On Sep 19, 6:05 pm, Robert Kern wrote: >> >http://nedbatchelder.com/text/python-parsers.html >> >> This is more a less just a list of parsers. I would like some detailed >> guidelines on which one to cho

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 19, 11:39 pm, TerryP wrote: [...] > For flat data, simple unix style rc or dos style ini file will often > suffice, and writing a parser is fairly trivial; in fact writing a [...] python already includes parsers for ".ini" configuration files. [...] > The best way to choose a parser, is e

Re: Comparison of parsers in python?

2009-09-20 Thread andrew cooke
On Sep 19, 9:34 pm, Peng Yu wrote: > On Sep 19, 6:05 pm, Robert Kern wrote: > >http://nedbatchelder.com/text/python-parsers.html > > This is more a less just a list of parsers. I would like some detailed > guidelines on which one to choose for various parsing problems. it would be simpler if you

Re: Comparison of parsers in python?

2009-09-19 Thread TerryP
Peng Yu wrote: > This is more a less just a list of parsers. I would like some detailed > guidelines on which one to choose for various parsing problems. > > Regards, > Peng It depends on the parsing problem. Obviously your not going to use an INI parser to work with XML, or vice versa. Likewise

Re: Comparison of parsers in python?

2009-09-19 Thread Peng Yu
On Sep 19, 6:05 pm, Robert Kern wrote: > Peng Yu wrote: > > Hi, > > > I did a google search and found various parser in python that can be > > used to parse different files in various situation. I don't see a page > > that summarizes and compares all the available parsers in python, from > > simpl

Re: Comparison of parsers in python?

2009-09-19 Thread Robert Kern
Peng Yu wrote: Hi, I did a google search and found various parser in python that can be used to parse different files in various situation. I don't see a page that summarizes and compares all the available parsers in python, from simple and easy-to-use ones to complex and powerful ones. Second

Comparison of parsers in python?

2009-09-19 Thread Peng Yu
Hi, I did a google search and found various parser in python that can be used to parse different files in various situation. I don't see a page that summarizes and compares all the available parsers in python, from simple and easy-to-use ones to complex and powerful ones. I am wondering if somebo