> I agree would be a serious changes, and I was thinking even of experimenting > a bit with this kind of strictly typed meta programming. It's perfectly > viable, as I've seen some good examples in my life. (and Template > Haskell does it AFAIR).
I'm not familiar with Template Haskell at all, but I don't think it does what you say. From what I understood of it, it was quite close from Camlp4, if maybe integrated to GHC in a tighter way (Camlp4 is really a separate source-to-source preprocessing phase). In particular, you can write and use an extension even if is produces ill-typed code in some circumstances (I mean type-checking the extension doesn't give you that the expansions will themselves be well-typed), and I haven't heard of getting type feedback at extension-writing time. I would welcome any pointer about this. > Of course worth to point out MetaOCaml and MetaML. They do runtime > type safe staged meta programming. Staged metaprogramming is indeed interesting but it is also very different from Camlp4. It is not a separate processing pass but really program generation at runtime. I don't think it would make much sense to try to integrate it into Camlp4, but I would indeed support such an addition to the OCaml language. Btw., there is an olg bugtracker entry about it, which I follow as a sign of support (maybe there would be a market for "I support feature request #0004608" stickers). It hasn't evolved much as I suppose nobody has time to work on it, and getting this to work reliably in both ocamlc and ocamlopt would certainly be an awful lot of work, but that's one case where "community" has helped bug triaging by expressing interest. http://caml.inria.fr/mantis/print_bug_page.php?bug_id=4608 > Good example would be: > let t = (1,2,3,4) in > map_tuple t (fun x -> x*2) I'm not sure this example works in MetaOCaml, or at least that map_tuple could be used on tuples of any width (of course you could have a "map_tuple" function only taking care of 4-uplets). I understand that you want a type-dependent metaprogramming layer to expression transformations that couldn't be typed in the source language. I think Alain Frisch dynamic type information injection proposal, plus in-language runtime computation, could be simpler for about the same expressive power. If you throw in MetaOCaml quotations, you can even remove the runtime interpretive costs. But who's going to work on that? I think that's orthogonal to syntactic processing anyway. > - syntax extension allows you to do certain things that are not available > in a normal way, to enumerate: list comprehensions, pa_lwt, pa_where, > pa_monad, pa_js. So they are clearly very useful in some cases If we had no Camlp4, we should push for some of these things to be integrated in the language. A reasonable but solid mixfix syntax could replace pa_monad, the "##"-syntax of pa_js, and some aspects of pa_lwt. "jsnew" and list comprehensions could perfectly be handled as quotations. I'm honored you mention mention pa_where (being one of the original authors, I'm happy to know that it brings happiness to your home in those Christmas times). I think that's also a good example of the dangers of Camlp4. The precedence rules for such a syntax extension are absolutely horrible to define and to get right; we have been bitten by countless bug writing it, and I still don't think it's safe for all uses. We would be better with either a solid version of it entering the language (... but it's hard to convince people that it's worth it, especially if you introduce new ambiguities opportunities), or maybe just forgetting the idea. > - combinators in particular monads allow certain syntactical > abstractions, however there are runtime costs because of constructing > closures and evaluating them later I personally dislike uses of Camlp4 to optimize code by rewriting some known expressions (eg. (... >>= (fun x -> ...)) into a more efficient form. It is fragile precisely because you can't get accurate points-to information so are unsure the operators you're manipulating really have the semantics corresponding to your rewrite. I think those things should be done in a different layer (not Camlp4), or not at all. Evidently other people have a different opinion, and that's fine. On Sun, Dec 11, 2011 at 1:47 AM, Wojciech Meyer <wojciech.me...@googlemail.com> wrote: > Gabriel Scherer <gabriel.sche...@gmail.com> writes: > >> A summary to this lengthy mail: >> (1) Why type-enriched Camlp4 is an unreasonable idea >> (2) We should extract the typedtree; why it's hard >> (3) A fictional narrative of the camlp4/camlp5 history >> (4) Why you don't want to become Camlp4 maintainer >> (5) How we could try not to use Camlp4 in the future >> (6) Syntax extension survival advices >> >> # (1) Why type-enriched Camlp4 is an unreasonable idea >> >> Wojciech, your idea of having type information at the Camlp4 level is >> absolutely unreasonable. You are not speaking about a "minor change" >> here, but a major rewrite that would affect the compiler internals as >> well. It would really be a new (and interesting) project. > > Hello Gabriel, > > I agree would be a serious changes, and I was thinking even of experimenting > a bit with this kind of strictly typed meta programming. It's perfectly > viable, as I've seen some good examples in my life. (and Template > Haskell does it AFAIR). > >> >> Camlp4 is, and I guess will remain, a syntax-level preprocessing >> tool. You have to accept the fact that you can't use type information >> at this level (but you can certainly "interact" in some way with the >> type system by producing/transforming pieces of code in a way that you >> know will have interesting typing effects; for example, you may want >> to generate code that is purposedly ill-typed in some cases, to >> disallow certain uses of your syntax extension). I'm not even sure >> what it would mean to access type information at the camlp4 level, as >> you're producing and transforming untyped AST; would you want >> partially typed ASTs? How is the typer supposed to work on the part >> that you haven't transformed yet, and therefore are not valid OCaml >> syntax? I suppose you could have a "preprocessing and transformation" >> tool at the typedtree level, but that would be a different tool with >> different uses, distinct from the syntactic preprocessing part (though >> you may develop "extensions" that act on both fronts). >> >> I'm not aware of so much Camlp4 situations that would really require >> typing information. I would be interested in good examples if you have >> some. One problem that I have had with Camlp4 is that you don't have >> identifier resolution information (eg. you don't know if the >> identifier "(@)" you're seeing is really list concatenation, or has >> been redefined/shadowed in the context); this makes uses of Camlp4 for >> inlining, for example, quirky and fragile. That's still a simpler >> problem than type information. > > Of course worth to point out MetaOCaml and MetaML. They do runtime > type safe staged meta programming. > > Good example would be: > let t = (1,2,3,4) in > map_tuple t (fun x -> x*2) > > (at this point I deliberately chosen closure and not additional syntax > extension because you can pass closures and you can't pass code easily > in Camlp4, because of the mentioned single stage meta programming). > > of course it's possible with dependent types and lists, but with meta > programming you could just expand map_tuple to the code you want, *but* > only when the type information is available. (or you can infer the type > yourself, in the simplest case if the tuple was expression passed > directly to map_tuple). > > Currently I believe that DSLs are the best way of achieving higher level > abstractions, because you are no longer bound to the language > syntax. And you can stay distant from the semantics of your target > language. > >> # (6) Syntax extension survival advices >> >> To the reader considering use of a new syntax extension in his next project: >> >> - don't > > Personally I disagree with this statement, because: > > - syntax extension allows you to do certain things that are not available > in a normal way, to enumerate: list comprehensions, pa_lwt, pa_where, > pa_monad, pa_js. So they are clearly very useful in some cases. > > - combinators in particular monads allow certain syntactical > abstractions, however there are runtime costs because of constructing > closures and evaluating them later. These runtime costs can easily > trimmed to some level with some compiler optimisations for instance > good inlining (that we don't have as mentioned before in the thread) and > efficiency of runtime (we do have a very good gc). > > - compilation is much easier than interpretation. You see the code > generated and the code generated is restricted semantically by the > target language. Handling environment is way lower level than just > generating some let bindings in a quotation. > > - >> >> - if you really must, try to make sure that your code is also >> reasonable to use *without* a syntax extension (eg. by producing >> a library with a clean interface, making your extension desugar to >> uses of it, but also making sure that it can be used by the human >> user) >> >> - if you really must, try to get it in the form of a quotation; the >> rest is fragile > > I agree that quotations are the best part of Camlp4. Right now I use > them to generate a lot of ML code (more than hundreds of KLOC) out of data > oriented DSL that parsing is detached from the official Camlp4 parser (I > use Menhir for that task). It works pretty well and the code generator > is pretty compact. > >> >> - alternatively, try to branch yourself on existing "flexible" syntax >> extensions such as Markus Mottl's 'type-conv' and Jeremy Yallop's >> 'patterns': you are relatively safe if you don't write any line of >> code modifying OCaml's syntax yourself. >> >> http://hg.ocaml.info/release/type-conv >> http://code.google.com/p/ocaml-patterns/ >> >> - do not hesitate to send for code review and/or ask for help on the >> list > > Additionally in case of if you need any help, I would volunteer to help. > > Cheers; > Wojciech -- Caml-list mailing list. Subscription management and archives: https://sympa-roc.inria.fr/wws/info/caml-list Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
