Re: [abcusers] proposal for developers: modular ABC
On Tue, 8 Jul 2003 12:41:08 +0200 (W. Europe Daylight Time), I. Oppenheim [EMAIL PROTECTED] wrote: It seems now that abcm2ps now also supports the ! kludge, so this could be a good starting point for the ABC parser library. [snip] I already proposed it some time ago! The advantages of my parser over the other abc2ps clones are: - it is clearly isolated in the 2 files abcparse.h and abcparse.c. - it may be used either for printing, editing or playing. There is no documentation (yet), but I hope the interface is self-explanatory enough for C programmers, and examples on how to use it may be found in abcm2ps and tclabc. About reading stdin instead of a file, I would say that this parser gets the tune from memory. This offers a quicker way to read the files (read(2) instead of gets(3), possible use of mmap(2)), and also permits parsing of static tunes (see tclabc on X11 selection paste). On the other side, reading from stdin asks for memory re-allocation... -- Ken ar c'hentañ | ** Breizh ha Linux atav! ** | http://moinejf.free.fr/ Pépé Jef| mailto:[EMAIL PROTECTED] To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
I. Oppenheim wrote: On Tue, 8 Jul 2003, T.M. Sommers wrote: Wil Macaulay wrote: If I'm not mistaken, you can also embed L:, M:, and K: in the tune proper, and I've seen N: and I: as well. But they can't be mistaken for notes. Yes, they can, since H-Z can be redefined. I guess it would depend, then, on how the substitution is done. Is it a straight text substitution, as in cpp, or is done at a later stage? This should probably be defined in the standard. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
I. Oppenheim wrote:
On Mon, 7 Jul 2003, T.M. Sommers wrote:
These are not problems with lex and yacc specifically; any program
parsing abc will have to face them. They are ambiguities, or potential
ambiguities, in the language itself.
In the first case above, there should be no problem: the composer field
is only in the header and starts a line, and the staff labels are in the
V field also in the header.
Contrary to what one would expect, this is more
problematic to deal with in Lex/Yacc than in other
parsers. The point is that Lex/Yacc do not have
facilities for context dependent tokenizing (are we in
a header or outside) nor a clean interface to exchange
information between the scanner and the parser. So one
would have to resort to global flags, which would make
the parser among other things thread-unsafe.
flex has context dependent tokenizing; it calls the contexts
'start conditions'. Since flex is free, and runs just about
everywhere, I don't see why a reference implementatin could not
use it.
In the second case (which is really a lexing issue), either disallow two
single bars next to each other (does such a construct even make sense?),
or require whitespace between them.
Well,
bar: '|' { $$.type = T_REG_BAR }
| '|' '|' { $$.type = T_DBL_BAR_1 }
;
Is all it takes to create a shift/reduce conflict in
Yacc.
'|' and '||' should be tokens, and distinguished in the lexer. C
manages to distinguish between them without any problems.
To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
Wil Macaulay wrote: If I'm not mistaken, you can also embed L:, M:, and K: in the tune proper, and I've seen N: and I: as well. But they can't be mistaken for notes. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
On Tue, 8 Jul 2003, T.M. Sommers wrote: Wil Macaulay wrote: If I'm not mistaken, you can also embed L:, M:, and K: in the tune proper, and I've seen N: and I: as well. But they can't be mistaken for notes. Yes, they can, since H-Z can be redefined. Irwin To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
It seems now that abcm2ps now also supports the ! kludge, so this could be a good starting point for the ABC parser library. Quoting from the abcm2ps change log: Version 3.6.4 - 03/07/06 Accept '!' as new line inside a music line (thanks to John Chambers). Groeten, Irwin Oppenheim [EMAIL PROTECTED] ~~~* Chazzanut Online: http://www.joods.nl/~chazzanut/ To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
I. Oppenheim writes: | On Tue, 8 Jul 2003, John Chambers wrote: | | An hour later, someone else's | request handles the job of purging the old files. | | Why don't you use a cron job for that? Because it doesn't always work. Over the years that the Tune Finder has existed, the MIT machine it's running on has been three different actual machines, with many more OS releases than that. At times, I've found that I don't have permission to use cron. It's also run on a couple of home machines (where I always have permission), and on a few other machines (mostly Toby Rider's, where permissions have also varied). Now granted, I can eventually get cron working. But this can take time, and often depends on someone else changing the permissions. When faced with the immediate problem of a disk filling up, it's a lot easier to do it in my own scripts. They always have access to their own tmp directory, and can always delete the files. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
On Tue, 8 Jul 2003, John Chambers wrote: | Why don't you use a cron job for that? Because it doesn't always work. Then what about this: #!/bin/sh # Will run once an hour and remove # all the temporary files older than # an hour in a given directory. # Provided as-is by Irwin Oppenheim. DIR=/home/jc/abc# the directory with temp files while : do touch $DIR/reference # store the current time sleep 3600 # be quiet for one hour for file in $DIR/*.tmp do if [ $file -ot $DIR/reference ] then rm -f $file fi done done # end of script Who needs cron :-) Just run it in the background, but make sure that you own the directory $DIR and that the sticky bit has not been set; otherwise the script might not be able to delete the files. Of course the webserver must be able to write in the directory in the first place. Groeten, Irwin Oppenheim [EMAIL PROTECTED] ~~~* Chazzanut Online: http://www.joods.nl/~chazzanut/ To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
On Mon, 7 Jul 2003, Bert Van Vreckem wrote: I. Oppenheim wrote: If the ABC community as a whole could provide a standard complying parser, not every developer that wishes to use ABC would have to reinvent the wheel again and again. The current situation is in nobody's interest. We're already there: using Henrik's BNF spec, a developer can generate a parser with tools like lex and yacc. Unfortunately, it is not trivial at all to convert a BNF definition into a working parser. 1/ the parser should be able to ignore future ABC extentions in a graceful way, whatever they may be; 2/ the parser should be able to understand (rather than ignore) non-standard legacy code; 3/ the parser should be able to give detailed error and warning messages (such as: warning: do not recognize x3/4, in line L, column C, rather than the typical: syntax error in line L); 4/ the parser should propagate parser events and data structures to the backend; 5/ the parser should be isolated in a module with a clear and well-defined API; 6/ the parser should never crash! When you use Lex and Yacc, as you suggested, you will come across all kind of non trivial technical problems. Just three examples to give you a taste: 1/ How can Lex know whether a C it scans is a note name, or the first letter of a staff label? Maybe it's even the beginning of the composer field! 2/ How can Yacc know whether || is one double bar symbol, or two single bar symbols next to each other? 3/ How can Lex/Yacc know whether E:| is a (mid-tune) field that starts with a pipe, or a note with a repeat sign next to it? I do not know whether Lex and Yacc are the proper tools to deal with ABC. I think these problems will make clear why it would be VERY useful to create a libabc parser as a group project. Groeten, Irwin Oppenheim [EMAIL PROTECTED] ~~~* Chazzanut Online: http://www.joods.nl/~chazzanut/ To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
I. Oppenheim wrote: On Mon, 7 Jul 2003, Bert Van Vreckem wrote: We're already there: using Henrik's BNF spec, a developer can generate a parser with tools like lex and yacc. Unfortunately, it is not trivial at all to convert a BNF definition into a working parser. snip When you use Lex and Yacc, as you suggested, you wil come across all kind of non trivial technical problems. Okay, perhaps I should first learn lex and yacc before ventilating opinions about it that are not backed by any factual knowledge whatsoever... ;-) I think these problems will make clear why it would be VERY useful to create a libabc parser as a group project. Well, a few years ago, I actually did a similar suggestion, but it didn't take off. Unfortunately, I'm not a C programmer. But if you are and you're willing, you're welcome at the abc project at Sourceforge. Send me your SF username and I'll add you to the project. This goes for anyone that wants to contribute, by the way. -- Bert Van Vreckem http://flanders.blackmill.net/ Not all chemicals are bad. Without chemicals such as hydrogen and oxygen, for example, there would be no way to make water, a vital ingredient in beer. -- Dave Barry To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
Richard Robinson writes: | I was thinking about this over the weekend ... Always a mistake. ;-) | I needes an abc parser last autumn, for my tunes-comparison project, | and really didn't want to write my own. So I used James Allwright's - | he uses it with different backends, to generate ps, midi abc, so it's | already separated out as necessary, all you need to do is supply the | event-handlers. But this created problems, since one of the bodies of music | I was most interested in looking at was the VMP, which mostly uses abc2win, | and it doesn't handle the ! linebreaks ... | | In short, I agree. A community parser would be a good and useful | thing. I was away for the weekend and am still catching up (you busy | people !) so I haven't got round to looking at John's code, to have any | idea how hard it would be to carve it out of the surrounding code, or | what sort sort of interface it would need, but I get the impression it | could be a good starting point. The liberal approach (handle what's out there) | would make it the most useful, plus that John's involved here. | | I'd be prepared to try help with such a project. What do you think, | John, would you support this ? Well, I'm not sure what support is needed. At first, it was easy to handle abc2win's use of !, since you could just treat it as a space. Then people came out with the !...! idea, and of course you'd want to handle those. So now you need a way of distinguishing the two meanings. I'm pretty sure that you won't find a good solution; the best is probably the kludge (i.e., heuristic) that I mentioned earlier. The abc2win ! could probably be included in a BNF syntax, but the result has a rather nasty ambiguity. So no matter what you do, you're facing either editing every abc2win file by hand, or writing a big of kludgery to distinguish the two uses. Actually, I do think this is a bit overblown. The kludge approach is a fairly small chunk of code, and seems to work pretty well. We can probably construct some pathological cases (like !fff!fff!), but they will be so rare in practice that we can just shrug and look the other way. I think the line-wrap problem is actually a bigger mess, since it produces illegal abc. One suggestion for a parser might be to bury this problem inside the input routine. It can easily implement the kludgery to detect the abc2win ! and replace it with a space. The resulting abc should be amenable to systematic parsing. I do wish that abc2win would itentify itself early in the headers. How pathological this can be was indicated in the recent example that contained a :|!: between two phrases. That's just pure perversity. I hope it was a bug, but I wouldn't put money on that. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
I. Oppenheim wrote: On Mon, 7 Jul 2003, Bert Van Vreckem wrote: I. Oppenheim wrote: If the ABC community as a whole could provide a standard complying parser, not every developer that wishes to use ABC would have to reinvent the wheel again and again. The current situation is in nobody's interest. We're already there: using Henrik's BNF spec, a developer can generate a parser with tools like lex and yacc. Unfortunately, it is not trivial at all to convert a BNF definition into a working parser. 1/ the parser should be able to ignore future ABC extentions in a graceful way, whatever they may be; 2/ the parser should be able to understand (rather than ignore) non-standard legacy code; It should also have an option that would disallow anything non-standard, such as gcc's -ansi switch. 1/ How can Lex know whether a C it scans is a note name, or the first letter of a staff label? Maybe it's even the beginning of the composer field! 2/ How can Yacc know whether || is one double bar symbol, or two single bar symbols next to each other? 3/ How can Lex/Yacc know whether E:| is a (mid-tune) field that starts with a pipe, or a note with a repeat sign next to it? I do not know whether Lex and Yacc are the proper tools to deal with ABC. These are not problems with lex and yacc specifically; any program parsing abc will have to face them. They are ambiguities, or potential ambiguities, in the language itself. In the first case above, there should be no problem: the composer field is only in the header and starts a line, and the staff labels are in the V field also in the header. In the second case (which is really a lexing issue), either disallow two single bars next to each other (does such a construct even make sense?), or require whitespace between them. In the third case, if E:| makes sense as a body field, then whitespace would be required to distinguish it from a note followed by a repeat, or perhaps better, do not allow a-g and A-G as body fields. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
On Mon, 7 Jul 2003, T.M. Sommers wrote:
These are not problems with lex and yacc specifically; any program
parsing abc will have to face them. They are ambiguities, or potential
ambiguities, in the language itself.
In the first case above, there should be no problem: the composer field
is only in the header and starts a line, and the staff labels are in the
V field also in the header.
Contrary to what one would expect, this is more
problematic to deal with in Lex/Yacc than in other
parsers. The point is that Lex/Yacc do not have
facilities for context dependent tokenizing (are we in
a header or outside) nor a clean interface to exchange
information between the scanner and the parser. So one
would have to resort to global flags, which would make
the parser among other things thread-unsafe.
In the second case (which is really a lexing issue), either disallow two
single bars next to each other (does such a construct even make sense?),
or require whitespace between them.
Well,
bar: '|' { $$.type = T_REG_BAR }
| '|' '|' { $$.type = T_DBL_BAR_1 }
;
Is all it takes to create a shift/reduce conflict in
Yacc.
or perhaps better, do not allow a-g and A-G as body fields.
Alas, they are in the standard.
Groeten,
Irwin Oppenheim
[EMAIL PROTECTED]
~~~*
Chazzanut Online:
http://www.joods.nl/~chazzanut/
To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
on 7/7/03 2:03 PM, John Chambers wrote: This is one of the reasons that I turned jcabc2ps into a normal unix filter that reads stdin if there are no input files. That way I can stick any sort of preprocessor on the beginning, and not worry about cleaning up intermediate files. Does abcm2ps work this way too? With the discussion on the list to merge the two efforts, this seems like a good first step. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
I.Oppenheim wrote: or perhaps better, do not allow a-g and A-G as body fields. Alas, they are in the standard. Of the capital letters only E: is allowed in the tune proper, and I think that's only meaningful in abc2mtex. The lower case letters have not been assigned. Phil Taylor To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
Tom Keays writes: | on 7/7/03 2:03 PM, John Chambers wrote: | This is one of the reasons that I turned jcabc2ps into a normal unix | filter that reads stdin if there are no input files. That way I can | stick any sort of preprocessor on the beginning, and not worry about | cleaning up intermediate files. | | Does abcm2ps work this way too? With the discussion on the list to merge | the two efforts, this seems like a good first step. Yeah, that was my first suggestion, a couple of days ago. I haven't heard back, but jef may be busy. (I know I am. ;-) This was one of the real disappointments that I had with abc2ps. With something like my tune finder, which uses it as the main tool for converting tunes to a list of formats, there is a real problem with managing all the intermediate files. It's very easy to end up with zillions of relic files on the disk. Unix's pipelines can be a real help in getting such messes under control. Actually, one of the things that some users of the tune finder's conversions have noticed and asked about: Sometimes when the job is done, there is a significant pause before the connection is broken and your browser says it's done. What's happening is the cleanup phase of the conversion script. It's searching the tmp directory for files more than an hour old, and deleting them. It's necessary to leave files around for a while, because browsers can sometimes take a while (i.e., more than a second) to get back and fetch things like images. The strategy is to not delete any file that may possibly be of some use in the near future, to give the client browser time to get everything. An hour later, someone else's request handles the job of purging the old files. Yes, it's a pain in the ass. ;-) And it's mostly because of all the programs that insist on using files rather than pipes. To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
If I'm not mistaken, you can also embed L:, M:, and K: in the tune proper, and I've seen N: and I: as well. wil Phil Taylor wrote: I.Oppenheim wrote: or perhaps better, do not allow a-g and A-G as body fields. Alas, they are in the standard. Of the capital letters only E: is allowed in the tune proper, and I think that's only meaningful in abc2mtex. The lower case letters have not been assigned. Phil Taylor To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
On Mon, Jul 07, 2003 at 12:16:42AM +0200, I. Oppenheim wrote: On Sun, 6 Jul 2003, Jeff Szuhay wrote: I'm thinking along the lines of an ABCParse module which feeds a stream to both ABCPlay module and ABCView modules. You're thinking about creating a libabc project? that would be very cool! Maybe we could form some group of developers that could isolate the existing parser from say the abcm2ps code and transform it into an independent library with a well-defined API? If the ABC community as a whole could provide a standard complying parser, not every developer that wishes to use ABC would have to reinvent the wheel again and again. The current situation is in nobody's interest. I was thinking about this over the weekend ... I needes an abc parser last autumn, for my tunes-comparison project, and really didn't want to write my own. So I used James Allwright's - he uses it with different backends, to generate ps, midi abc, so it's already separated out as necessary, all you need to do is supply the event-handlers. But this created problems, since one of the bodies of music I was most interested in looking at was the VMP, which mostly uses abc2win, and it doesn't handle the ! linebreaks ... In short, I agree. A community parser would be a good and useful thing. I was away for the weekend and am still catching up (you busy people !) so I haven't got round to looking at John's code, to have any idea how hard it would be to carve it out of the surrounding code, or what sort sort of interface it would need, but I get the impression it could be a good starting point. The liberal approach (handle what's out there) would make it the most useful, plus that John's involved here. I'd be prepared to try help with such a project. What do you think, John, would you support this ? We've been here before ... of course, not everybody uses C. And so on ... But it would still be more help than hindrance. And then, of course, it should be possible to produce bindings to perl, etc ... Maybe this is the wrong list (I never joined the developer list) Is there still another ABC list? It's all right, the developer's list died a long time ago. -- Richard Robinson The whole plan hinged upon the natural curiosity of potatoes - S. Lem To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
On Sun, 6 Jul 2003, Jeff Szuhay wrote: I'm thinking along the lines of an ABCParse module which feeds a stream to both ABCPlay module and ABCView modules. You're thinking about creating a libabc project? that would be very cool! Maybe we could form some group of developers that could isolate the existing parser from say the abcm2ps code and transform it into an independent library with a well-defined API? If the ABC community as a whole could provide a standard complying parser, not every developer that wishes to use ABC would have to reinvent the wheel again and again. The current situation is in nobody's interest. Maybe this is the wrong list (I never joined the developer list) Is there still another ABC list? Groeten, Irwin Oppenheim [EMAIL PROTECTED] ~~~* Chazzanut Online: http://www.joods.nl/~chazzanut/ To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
Re: [abcusers] proposal for developers: modular ABC
I. Oppenheim wrote: If the ABC community as a whole could provide a standard complying parser, not every developer that wishes to use ABC would have to reinvent the wheel again and again. The current situation is in nobody's interest. We're already there: using Henrik's BNF spec, a developer can generate a parser with tools like lex and yacc. -- Bert Van Vreckem http://flanders.blackmill.net/ Not all chemicals are bad. Without chemicals such as hydrogen and oxygen, for example, there would be no way to make water, a vital ingredient in beer. -- Dave Barry To subscribe/unsubscribe, point your browser to: http://www.tullochgorm.com/lists.html
