Thank you for clarifying this, Matthias! 2017-10-25 23:48 GMT+03:00 Matthias Boehm <mboe...@googlemail.com>:
> Hi Nantia, > > sure, let me clarify these two statements: > > 1) Optimization level: When I said O1, I was referring to SystemML's > configurable optimization level 1, which can be set through > 'sysml.optlevel' (or 'optlevel' in SystemML 0.14) in a SystemML-config.xml > file or via setConfigProperty in MLContext. By default we use O2 with all > rewrites enabled, in O1, certain rewrites are disabled, which allows > comparing plans with and without rewrites. Furthermore, the quoted > statement below was more of a note for myself, to cleanup this handling of > remaining rewrites (that are not conditionally disabled yet) in an upcoming > SystemML release. > > 2) Log4j Configuration: You can take the log4j properties file from our > conf directory, put it into your classpath, and append the following line > to this file (alternatively with TRACE instead of DEBUG if you want to see > more details): > log4j.logger.org.apache.sysml.hops.rewrite=DEBUG > > Regards, > Matthias > > On Wed, Oct 25, 2017 at 4:24 AM, Nantia Makrynioti <nantiam...@gmail.com> > wrote: > > > Hello Matthias, > > > > Thanks a lot for replying and sorry for my late response. > > > > I have two more questions regarding the steps you described. > > > > I think it's a > > > good idea to clean this up (disable these remaining rewrites in O1 as > > > well), which would be useful for debugging. > > > > > > I don't understand what you mean by O1. So how can I disable these > > remaining rewrites? > > > > If you're debugging any specific issue and want to see which rewrites > > > trigger where, you can set the log level for package > > > 'org.apache.sysml.hops.rewrite' to DEBUG or TRACE in your log4j > > > configuration. > > > > > > I am working with SystemML 0.14.0. There is a log4j.properties file, but > I > > cannot find anything relevant to 'org.apache.sysml.hops.rewrite'. Is > > there > > another file I should check? > > > > Thanks again, > > Nantia > > > > 2017-10-13 23:29 GMT+03:00 Matthias Boehm <mboe...@googlemail.com>: > > > > > Hi Nantia, > > > > > > in optimization level 1, we disable the following rewrites and the > > explain > > > hops or runtime output will show the resulting plan: > > > * Disable common-subexpression elimination > > > * Disable algebraic simplifications (static and dynamic) > > > * Disable inter-procedural analysis > > > * Disable branch removal and statement block merge > > > * Disable sum-product rewrites > > > * Disable update-in-place rewrites > > > > > > So, to compare plans without and with rewrites, you would need to run > > with > > > optimization level 1 and 2 (our default), and capture & compare these > > > outputs. > > > > > > Note that there are a number of rewrites such as matrix multiplication > > > chain optimization, operator selection, hop-lop rewrites that are > always > > > applied and thus not affected by the optimization level. I think it's a > > > good idea to clean this up (disable these remaining rewrites in O1 as > > > well), which would be useful for debugging. > > > > > > If you're debugging any specific issue and want to see which rewrites > > > trigger where, you can set the log level for package > > > 'org.apache.sysml.hops.rewrite' to DEBUG or TRACE in your log4j > > > configuration. If you're compiling from sources, you can also simply > set > > > ProgramRewriter.LDEBUG to true to do the same thing. > > > > > > Regards, > > > Matthias > > > > > > On Fri, Oct 13, 2017 at 2:04 AM, Nantia Makrynioti < > nantiam...@gmail.com > > > > > > wrote: > > > > > > > Hello, > > > > > > > > I set optimization level to 1 in SystemML-config.xml, in order to get > > HOP > > > > and LOP plans before and after rewrites. However, I am still getting > > > just a > > > > single plan. > > > > > > > > I am using spark shell to execute the dml script. > > > > > > > > Thank you in advance, > > > > Nantia > > > > > > > > > >
