Re: [racket-dev] Demodularizer
I just pushed a few commits that should speed it up a bit. Jay On Fri, Oct 29, 2010 at 10:50 PM, Jon Rafkind rafk...@cs.utah.edu wrote: Ok nevermind about the bug, I had some leftover .zo files. When I erased my compiled directory and reran the demodularizer it worked. Another quick stat: $ time raco demod p.rkt real 3m55.563s 1.8ghz amd On 10/29/2010 10:40 PM, Jon Rafkind wrote: How do you run the resulting _merged file? I ran 'raco demod p.rkt' and got compiled/p_rkt_zo_merged.zo. Then I tried running racket on it but got an error $ ~/bin/plt/bin/racket p_rkt_zo_merged compiled/p_rkt_zo_merged.zo: read (compiled): ill-formed code (bad count: 436459 != 801349, started at 11951) Also just some quick stats: $ cat p.rkt #lang racket (define (hello) (printf Hello world!)) (hello) $ ls -l compiled/p_rkt.zo ;; just using raco make 352 $ ls -l compiled/p_rkt_zo_merged.zo 448410 Would you mind renaming the command to 'demodularize' ? On 10/29/2010 07:53 PM, Jay McCarthy wrote: Here is a message from Blake Johnson about something I just pushed for him: Jay recently committed my changes implementing raco demod, which will flatten a modular program into a single compiled module. How to use it: Run raco demod filename. This will produce a demodularized zo file called filename_merged.zo which can then be run by passing it to racket. This process can take quite a while, so if you want to see what is going on, you can set PLTSTDERR=debug or info. Once the zo file is created, it can be moved and run from anywhere because it has no dependencies. What it doesn't do (yet): - Dead code elimination This is partially implemented but doesn't handle every case properly, so it is turned off for now. - Optimize the program This is the next big thing I'm working on, which involves decompiling the zo slightly and converting it into c structs so that the existing optimizer can run on it. - Work with programs that dynamically manipulate modules This means things like DrRacket won't be able to be demodularized. We have some ideas on how to handle this, but it probably won't happen any time soon. Other improvements: zo-parse and zo-marshal should be able to handle any .zo you throw at them. Help needed: Any programs that don't work after demodularization would be helpful. Thanks, Blake Johnson At the moment, we have not measured any performance improvements, etc after using the demodularizer. At the moment, we do not anticipate that there will be any because we are not doing DCE or the re-optimization. We'll let everyone know when we measure that. For the moment, this may be of particular interest to any out there that are doing program analysis and would like whole programs. Jay _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://teammccarthy.org/jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Demodularizer
Just for some numbers, http://drdr.racket-lang.org/21386/collects/tests/compiler/demodularizer/ used to take 5.76 minutes and now it takes 2.90 Jay On Sat, Oct 30, 2010 at 9:17 AM, Jay McCarthy jay.mccar...@gmail.com wrote: I just pushed a few commits that should speed it up a bit. Jay On Fri, Oct 29, 2010 at 10:50 PM, Jon Rafkind rafk...@cs.utah.edu wrote: Ok nevermind about the bug, I had some leftover .zo files. When I erased my compiled directory and reran the demodularizer it worked. Another quick stat: $ time raco demod p.rkt real 3m55.563s 1.8ghz amd On 10/29/2010 10:40 PM, Jon Rafkind wrote: How do you run the resulting _merged file? I ran 'raco demod p.rkt' and got compiled/p_rkt_zo_merged.zo. Then I tried running racket on it but got an error $ ~/bin/plt/bin/racket p_rkt_zo_merged compiled/p_rkt_zo_merged.zo: read (compiled): ill-formed code (bad count: 436459 != 801349, started at 11951) Also just some quick stats: $ cat p.rkt #lang racket (define (hello) (printf Hello world!)) (hello) $ ls -l compiled/p_rkt.zo ;; just using raco make 352 $ ls -l compiled/p_rkt_zo_merged.zo 448410 Would you mind renaming the command to 'demodularize' ? On 10/29/2010 07:53 PM, Jay McCarthy wrote: Here is a message from Blake Johnson about something I just pushed for him: Jay recently committed my changes implementing raco demod, which will flatten a modular program into a single compiled module. How to use it: Run raco demod filename. This will produce a demodularized zo file called filename_merged.zo which can then be run by passing it to racket. This process can take quite a while, so if you want to see what is going on, you can set PLTSTDERR=debug or info. Once the zo file is created, it can be moved and run from anywhere because it has no dependencies. What it doesn't do (yet): - Dead code elimination This is partially implemented but doesn't handle every case properly, so it is turned off for now. - Optimize the program This is the next big thing I'm working on, which involves decompiling the zo slightly and converting it into c structs so that the existing optimizer can run on it. - Work with programs that dynamically manipulate modules This means things like DrRacket won't be able to be demodularized. We have some ideas on how to handle this, but it probably won't happen any time soon. Other improvements: zo-parse and zo-marshal should be able to handle any .zo you throw at them. Help needed: Any programs that don't work after demodularization would be helpful. Thanks, Blake Johnson At the moment, we have not measured any performance improvements, etc after using the demodularizer. At the moment, we do not anticipate that there will be any because we are not doing DCE or the re-optimization. We'll let everyone know when we measure that. For the moment, this may be of particular interest to any out there that are doing program analysis and would like whole programs. Jay _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://teammccarthy.org/jay The glory of God is Intelligence - DC 93 -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://teammccarthy.org/jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
[racket-dev] Demodularizer
Here is a message from Blake Johnson about something I just pushed for him: Jay recently committed my changes implementing raco demod, which will flatten a modular program into a single compiled module. How to use it: Run raco demod filename. This will produce a demodularized zo file called filename_merged.zo which can then be run by passing it to racket. This process can take quite a while, so if you want to see what is going on, you can set PLTSTDERR=debug or info. Once the zo file is created, it can be moved and run from anywhere because it has no dependencies. What it doesn't do (yet): - Dead code elimination This is partially implemented but doesn't handle every case properly, so it is turned off for now. - Optimize the program This is the next big thing I'm working on, which involves decompiling the zo slightly and converting it into c structs so that the existing optimizer can run on it. - Work with programs that dynamically manipulate modules This means things like DrRacket won't be able to be demodularized. We have some ideas on how to handle this, but it probably won't happen any time soon. Other improvements: zo-parse and zo-marshal should be able to handle any .zo you throw at them. Help needed: Any programs that don't work after demodularization would be helpful. Thanks, Blake Johnson At the moment, we have not measured any performance improvements, etc after using the demodularizer. At the moment, we do not anticipate that there will be any because we are not doing DCE or the re-optimization. We'll let everyone know when we measure that. For the moment, this may be of particular interest to any out there that are doing program analysis and would like whole programs. Jay -- Jay McCarthy j...@cs.byu.edu Assistant Professor / Brigham Young University http://teammccarthy.org/jay The glory of God is Intelligence - DC 93 _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Demodularizer
How do you run the resulting _merged file? I ran 'raco demod p.rkt' and got compiled/p_rkt_zo_merged.zo. Then I tried running racket on it but got an error $ ~/bin/plt/bin/racket p_rkt_zo_merged compiled/p_rkt_zo_merged.zo: read (compiled): ill-formed code (bad count: 436459 != 801349, started at 11951) Also just some quick stats: $ cat p.rkt #lang racket (define (hello) (printf Hello world!)) (hello) $ ls -l compiled/p_rkt.zo ;; just using raco make 352 $ ls -l compiled/p_rkt_zo_merged.zo 448410 Would you mind renaming the command to 'demodularize' ? On 10/29/2010 07:53 PM, Jay McCarthy wrote: Here is a message from Blake Johnson about something I just pushed for him: Jay recently committed my changes implementing raco demod, which will flatten a modular program into a single compiled module. How to use it: Run raco demod filename. This will produce a demodularized zo file called filename_merged.zo which can then be run by passing it to racket. This process can take quite a while, so if you want to see what is going on, you can set PLTSTDERR=debug or info. Once the zo file is created, it can be moved and run from anywhere because it has no dependencies. What it doesn't do (yet): - Dead code elimination This is partially implemented but doesn't handle every case properly, so it is turned off for now. - Optimize the program This is the next big thing I'm working on, which involves decompiling the zo slightly and converting it into c structs so that the existing optimizer can run on it. - Work with programs that dynamically manipulate modules This means things like DrRacket won't be able to be demodularized. We have some ideas on how to handle this, but it probably won't happen any time soon. Other improvements: zo-parse and zo-marshal should be able to handle any .zo you throw at them. Help needed: Any programs that don't work after demodularization would be helpful. Thanks, Blake Johnson At the moment, we have not measured any performance improvements, etc after using the demodularizer. At the moment, we do not anticipate that there will be any because we are not doing DCE or the re-optimization. We'll let everyone know when we measure that. For the moment, this may be of particular interest to any out there that are doing program analysis and would like whole programs. Jay _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
Re: [racket-dev] Demodularizer
Ok nevermind about the bug, I had some leftover .zo files. When I erased my compiled directory and reran the demodularizer it worked. Another quick stat: $ time raco demod p.rkt real 3m55.563s 1.8ghz amd On 10/29/2010 10:40 PM, Jon Rafkind wrote: How do you run the resulting _merged file? I ran 'raco demod p.rkt' and got compiled/p_rkt_zo_merged.zo. Then I tried running racket on it but got an error $ ~/bin/plt/bin/racket p_rkt_zo_merged compiled/p_rkt_zo_merged.zo: read (compiled): ill-formed code (bad count: 436459 != 801349, started at 11951) Also just some quick stats: $ cat p.rkt #lang racket (define (hello) (printf Hello world!)) (hello) $ ls -l compiled/p_rkt.zo ;; just using raco make 352 $ ls -l compiled/p_rkt_zo_merged.zo 448410 Would you mind renaming the command to 'demodularize' ? On 10/29/2010 07:53 PM, Jay McCarthy wrote: Here is a message from Blake Johnson about something I just pushed for him: Jay recently committed my changes implementing raco demod, which will flatten a modular program into a single compiled module. How to use it: Run raco demod filename. This will produce a demodularized zo file called filename_merged.zo which can then be run by passing it to racket. This process can take quite a while, so if you want to see what is going on, you can set PLTSTDERR=debug or info. Once the zo file is created, it can be moved and run from anywhere because it has no dependencies. What it doesn't do (yet): - Dead code elimination This is partially implemented but doesn't handle every case properly, so it is turned off for now. - Optimize the program This is the next big thing I'm working on, which involves decompiling the zo slightly and converting it into c structs so that the existing optimizer can run on it. - Work with programs that dynamically manipulate modules This means things like DrRacket won't be able to be demodularized. We have some ideas on how to handle this, but it probably won't happen any time soon. Other improvements: zo-parse and zo-marshal should be able to handle any .zo you throw at them. Help needed: Any programs that don't work after demodularization would be helpful. Thanks, Blake Johnson At the moment, we have not measured any performance improvements, etc after using the demodularizer. At the moment, we do not anticipate that there will be any because we are not doing DCE or the re-optimization. We'll let everyone know when we measure that. For the moment, this may be of particular interest to any out there that are doing program analysis and would like whole programs. Jay _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev _ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/dev
