[webkit-dev] Rewriting binding code generator, maybe?
Niwa, hi, CodeGenerator has already been rewritten in python, combining the following: * Mozilla's xpcom xpidl.py generator * gobject's codegen.py The following work was done: - The xpidl generator was modified to understand Webkit's IDL file format - gobject's codegen.py was modified to understand xpidl.py's AST object format - gobject codegen.py's output formatter was modified to understand both webkit datatypes and to output python It should be a trivial job to work out how to modify the code to output other generators. The only thing to watch out for is that it was a very fast hack put together quite quickly and there are unused parts (which will never be used) that still have gobject codegen output. Functionally however it works perfectly. By contrast, attempting to modify CodeGeneratorJS.pm to comprehend gobject took over eight weeks. The python-based code-generator was completed in under six. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Rewriting binding code generator, maybe?
Apologies, the link is here: http://git.savannah.gnu.org/cgit/pythonwebkit.git/tree/pywebkitgtk/wkcodegen?h=python_codegen ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
I think the best way to make such points is to create a bug and post a patch. -Filip On Jun 11, 2013, at 1:32 PM, wkcensorshipbypass00...@mailinator.com wrote: Niwa, hi, CodeGenerator has already been rewritten in python, combining the following: * Mozilla's xpcom xpidl.py generator * gobject's codegen.py The following work was done: - The xpidl generator was modified to understand Webkit's IDL file format - gobject's codegen.py was modified to understand xpidl.py's AST object format - gobject codegen.py's output formatter was modified to understand both webkit datatypes and to output python It should be a trivial job to work out how to modify the code to output other generators. The only thing to watch out for is that it was a very fast hack put together quite quickly and there are unused parts (which will never be used) that still have gobject codegen output. Functionally however it works perfectly. By contrast, attempting to modify CodeGeneratorJS.pm to comprehend gobject took over eight weeks. The python-based code-generator was completed in under six. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
I could have sworn I saw someone in another thread post that they were interested in taking over maintenance of the binding generators (not necessarily relating to a massive rewrite, just for ongoing maintenance, was my impression). But I can't find that email at all now. Does anyone know who I'm thinking of? I'm interested in discussing refactoring of the binding generators at the contributors meeting, since the one time I had to hack on them (quite a while ago) I found it quite painful. - This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Thu, Apr 11, 2013 at 1:36 AM, Eric Seidel e...@webkit.org wrote: I know some folks in our TOK office have expressed strong interest in re-writing it in python for Blink. Perhaps there is an opportunity for some x-project collaboration here. Would be great if both projects could share the same generators. People writing device API bindings for web runtimes that are injected at runtime could also use it. Would be interesting if we could have the bindings generators as a separated/standalone project. Cheers, ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Apr 11, 2013, at 5:02 AM, Thiago Marcos P. Santos tmpsan...@gmail.com wrote: Would be great if both projects could share the same generators. The part that would be OK to share would be the parsing. The actual code generation needs to change all the time as the internal architecture of WebKit and of the JavaScript engine and other things we’re binding change, so we should not try to change that. People writing device API bindings for web runtimes that are injected at runtime could also use it. I understand, but I think there are different requirements there. For these things injected at runtime we’d want the API to be stable, whereas for inside the project we want the best possible performance even if many implementation details are exposed. WebKitTestRunner does something like this, and do I think that it has requirements very similar to other “injected at runtime” uses. Would be interesting if we could have the bindings generators as a separated/standalone project. That does not sound like a good idea to me for the specific reasons I mentioned above. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Apr 11, 2013, at 9:51 AM, Darin Adler da...@apple.com wrote: The actual code generation needs to change all the time as the internal architecture of WebKit and of the JavaScript engine and other things we’re binding change, so we should not try to change that. I meant “we should not try to share that or have it live outside the main project”. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Thu, Apr 11, 2013 at 9:51 AM, Darin Adler da...@apple.com wrote: On Apr 11, 2013, at 5:02 AM, Thiago Marcos P. Santos tmpsan...@gmail.com wrote: Would be great if both projects could share the same generators. The part that would be OK to share would be the parsing. The actual code generation needs to change all the time as the internal architecture of WebKit and of the JavaScript engine and other things we’re binding change, so we should not try to change that. I'm not sure how useful a suggestion this is, but ANTLR has a pretty strong framework for separating parsing from code generation and seems like it would be an ideal fit for this, except that the tool itself is written in Java. Perhaps that is a show-stopper? I'm not actually aware of any similar parser-generator tools that have similar architectures but are in more WebKit/Blink-friendly languages, but maybe they exist? It certainly would be nice not to have to roll your own framework here. -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Thu, Apr 11, 2013 at 10:12 AM, Dirk Pranke dpra...@chromium.org wrote: On Thu, Apr 11, 2013 at 9:51 AM, Darin Adler da...@apple.com wrote: On Apr 11, 2013, at 5:02 AM, Thiago Marcos P. Santos tmpsan...@gmail.com wrote: Would be great if both projects could share the same generators. The part that would be OK to share would be the parsing. The actual code generation needs to change all the time as the internal architecture of WebKit and of the JavaScript engine and other things we’re binding change, so we should not try to change that. I'm not sure how useful a suggestion this is, but ANTLR has a pretty strong framework for separating parsing from code generation and seems like it would be an ideal fit for this, except that the tool itself is written in Java. Perhaps that is a show-stopper? I'm not actually aware of any similar parser-generator tools that have similar architectures but are in more WebKit/Blink-friendly languages, but maybe they exist? It certainly would be nice not to have to roll your own framework here. We used to use Bison for CSS parsing and we might still use it somewhere. Having said that, C/C++ might be an overkill for writing a binding code generator. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Apr 11, 2013, at 10:12 AM, Dirk Pranke dpra...@chromium.org wrote: I'm not sure how useful a suggestion this is, but ANTLR has a pretty strong framework for separating parsing from code generation and seems like it would be an ideal fit for this… Speaking of other tools, this might be a nice use for SWIG (http://www.swig.org). It could generate Objective-C bindings out of the box, and it looks like someone started binding it to JavaScriptCore (https://github.com/Neha03/gsoc2012-javascript/tree/master/Lib/javascript/jsc). -Tim ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
dpranke, On 04/11/2013 02:12 PM, Dirk Pranke wrote: I'm not sure how useful a suggestion this is, but ANTLR has a pretty strong framework for separating parsing from code generation and seems like it would be an ideal fit for this, except that the tool itself is written in Java. Perhaps that is a show-stopper? I'm not actually aware of any similar parser-generator tools that have similar architectures but are in more WebKit/Blink-friendly languages, but maybe they exist? Pyparsing is a nice little library that requires no code generation (grammar is written in pure Python) and is very simple to use. Their website lives at http://pyparsing.wikispaces.com/. Leandro ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
I retract my earlier statement. I think that the constraints here between WebKit and blink are different enough that we should only re-examine sharing code after both projects have had a chance to re-write this subsystem to better fit their needs. We originally imported these bindings generators from KDOM (and changed them quite a bit), but I suspect that they were never really a perfect fit for WebKit in the first place. Since we're both theoretically parsing WebIDL here, presumably there are several good parsers already. If both WebKit and Blink choose the same language to re-write their respective bindings generators in, perhaps we'll end up using the same parser library. Best of luck! On Wed, Apr 10, 2013 at 3:36 PM, Eric Seidel e...@webkit.org wrote: I know some folks in our TOK office have expressed strong interest in re-writing it in python for Blink. Perhaps there is an opportunity for some x-project collaboration here. On Wed, Apr 10, 2013 at 3:32 PM, Ryosuke Niwa rn...@webkit.org wrote: Hi, Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I'm not necessarily volunteering to do the work myself at this moment but I wanted to see if there is any interest in this idea or not. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Thu, Apr 11, 2013 at 10:29 AM, Leandro Pereira lean...@profusion.mobiwrote: dpranke, On 04/11/2013 02:12 PM, Dirk Pranke wrote: I'm not sure how useful a suggestion this is, but ANTLR has a pretty strong framework for separating parsing from code generation and seems like it would be an ideal fit for this, except that the tool itself is written in Java. Perhaps that is a show-stopper? I'm not actually aware of any similar parser-generator tools that have similar architectures but are in more WebKit/Blink-friendly languages, but maybe they exist? Pyparsing is a nice little library that requires no code generation (grammar is written in pure Python) and is very simple to use. Their website lives at http://pyparsing.wikispaces.**com/http://pyparsing.wikispaces.com/ . That's pretty much the opposite of what I was getting at (I was talking about ANTLR's use of StringTemplate to separate code generation from the building of the AST). -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Rewriting binding code generator, maybe?
Hi, Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I'm not necessarily volunteering to do the work myself at this moment but I wanted to see if there is any interest in this idea or not. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
I know some folks in our TOK office have expressed strong interest in re-writing it in python for Blink. Perhaps there is an opportunity for some x-project collaboration here. On Wed, Apr 10, 2013 at 3:32 PM, Ryosuke Niwa rn...@webkit.org wrote: Hi, Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I'm not necessarily volunteering to do the work myself at this moment but I wanted to see if there is any interest in this idea or not. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
I know some folks in our TOK office have expressed strong interest in re-writing it in python for Blink. Yeah, I'm planning (but in low priority) to rewrite it in Python after cleaning up IDLs and code generators. The difficulty is that CodeGenerator*.pm are not the only target for rewriting. We also have to rewrite IDLParser.pm (2500 lines) and all other related Perl scripts at a breath. Although we can proceed with the work incrementally by defining intermediate data structure between Python and Perl, it will anyway be a huge amount of work. On Thu, Apr 11, 2013 at 7:36 AM, Eric Seidel e...@webkit.org wrote: I know some folks in our TOK office have expressed strong interest in re-writing it in python for Blink. Perhaps there is an opportunity for some x-project collaboration here. On Wed, Apr 10, 2013 at 3:32 PM, Ryosuke Niwa rn...@webkit.org wrote: Hi, Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I'm not necessarily volunteering to do the work myself at this moment but I wanted to see if there is any interest in this idea or not. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev -- Kentaro Hara, Tokyo, Japan https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Apr 10, 2013, at 6:32 PM, Ryosuke Niwa rn...@webkit.org wrote: Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. They can't be merged. We also have CodeGeneratorObjC.pm. (And internally we have CodeGeneratorSafari.pm.) I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I've also seen people express their frustration for our tools currently written in Ruby and Python. I personally find Perl fine for this task and don't see a need to rewrite things just because. — Timothy Hatcher ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
As a person who does not know any of Perl, Python or Ruby very well, I find unfamiliar Python the easiest to understand (even though I have more experience with Perl). I feel this would be worthwhile, if not necessarily a top priority. I do hope that this could be done without undue increase in verbosity. - Maciej On Apr 10, 2013, at 3:32 PM, Ryosuke Niwa rn...@webkit.org wrote: Hi, Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I'm not necessarily volunteering to do the work myself at this moment but I wanted to see if there is any interest in this idea or not. - R. Niwa ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Wed, Apr 10, 2013 at 4:59 PM, Timothy Hatcher timo...@apple.com wrote: On Apr 10, 2013, at 6:32 PM, Ryosuke Niwa rn...@webkit.org wrote: Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. They can't be merged. We also have CodeGeneratorObjC.pm. (And internally we have CodeGeneratorSafari.pm.) I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I've also seen people express their frustration for our tools currently written in Ruby and Python. I personally find Perl fine for this task and don't see a need to rewrite things just because. I'm sure Perl is a fine tool for the job, if you happen to know Perl, but Perl code is generally much less approachable than Ruby, and Ruby less so than Python. Of course many if not most of the Python zealots have gone elsewhere :). I believe in Blink-land there may be a reasonably concerted effort to move the stuff Chromium still uses to Python where possible, and obviously this is one such case (as Eric alludes to earlier), so it does seem like it would be nice to be able share the code and make the rewrite generic if it wasn't too much additional work, there was interest in doing so, and the rewrite was gonna happen anyway. Otherwise I'm quite sympathetic to the ain't broke argument here :) -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
I've seen many people expressing their inability to improve the binding code because of its being written in Perl. Although I agree that the complexity of the binding code partly comes from the fact that it's written in Perl, another big reason is that current code generators are more complicated than necessary. Now that JSC is the only engine in WebKit and V8 is the only engine in V8, we both can remove a bunch of unnecessary abstractions and IDL attributes that had existed to share code between JSC and V8. In short-term, this kind of clean-up will improve our situation a lot. (I'm doing the work with the highest priority in the V8 side.) On Thu, Apr 11, 2013 at 9:08 AM, Dirk Pranke dpra...@chromium.org wrote: On Wed, Apr 10, 2013 at 4:59 PM, Timothy Hatcher timo...@apple.comwrote: On Apr 10, 2013, at 6:32 PM, Ryosuke Niwa rn...@webkit.org wrote: Can we rewrite CodeGenerator*.pm in Ruby or Python? I feel that the current code is very hard to understand and hack on. In particular we have CodeGenerator.pm and CodeGeneratorJS.pm (CodeGeneratorV8.pm has been removed), and we need to merge them anyway. They can't be merged. We also have CodeGeneratorObjC.pm. (And internally we have CodeGeneratorSafari.pm.) I've seen many people expressing their inability to improve the binding code because of its being written in Perl. I've also seen people express their frustration for our tools currently written in Ruby and Python. I personally find Perl fine for this task and don't see a need to rewrite things just because. I'm sure Perl is a fine tool for the job, if you happen to know Perl, but Perl code is generally much less approachable than Ruby, and Ruby less so than Python. Of course many if not most of the Python zealots have gone elsewhere :). I believe in Blink-land there may be a reasonably concerted effort to move the stuff Chromium still uses to Python where possible, and obviously this is one such case (as Eric alludes to earlier), so it does seem like it would be nice to be able share the code and make the rewrite generic if it wasn't too much additional work, there was interest in doing so, and the rewrite was gonna happen anyway. Otherwise I'm quite sympathetic to the ain't broke argument here :) -- Dirk ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev -- Kentaro Hara, Tokyo, Japan https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
On Apr 10, 2013, at 8:17 PM, Kentaro Hara hara...@chromium.org wrote: Now that JSC is the only engine in WebKit and V8 is the only engine in V8, we both can remove a bunch of unnecessary abstractions and IDL attributes that had existed to share code between JSC and V8. In short-term, this kind of clean-up will improve our situation a lot. (I'm doing the work with the highest priority in the V8 side.) As I said in a previous email, JSC isn't the only binding we generate with these scripts. We also generate ObjC and internal Safari bindings. So those needs need to be considered for anyone rewriting these scripts. — Timothy Hatcher ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Rewriting binding code generator, maybe?
If a rewrite does happen, it would be nice to have some generic functionality abstracted away from the specific language generators. For example, the function to add an #include statement to the generated implementation file is currently replicated for every language. JS and V8 have AddToImplIncludes, while CPP and ObjC have AddIncludesForType. I personally copy+pasted a lot of code from CPP and JS for the Ruby code generator, and it would be nice if some of that was more generic. -Tim On Apr 10, 2013, at 6:03 PM, Timothy Hatcher timo...@apple.com wrote: On Apr 10, 2013, at 8:17 PM, Kentaro Hara hara...@chromium.org wrote: Now that JSC is the only engine in WebKit and V8 is the only engine in V8, we both can remove a bunch of unnecessary abstractions and IDL attributes that had existed to share code between JSC and V8. In short-term, this kind of clean-up will improve our situation a lot. (I'm doing the work with the highest priority in the V8 side.) As I said in a previous email, JSC isn't the only binding we generate with these scripts. We also generate ObjC and internal Safari bindings. So those needs need to be considered for anyone rewriting these scripts. — Timothy Hatcher ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
