On Thu, Jan 25, 2018 at 9:40 AM, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: > As far as I understand generation of native code is now always done for all > supported expressions and individually by each backend. > I wonder it will be useful to do more efforts to understand when compilation > to native code should be done and when interpretation is better. > For example many JIT-able languages like Lua are using traces, i.e. query is > first interpreted and trace is generated. If the same trace is followed > more than N times, then native code is generated for it. > > In context of DBMS executor it is obvious that only frequently executed or > expensive queries have to be compiled. > So we can use estimated plan cost and number of query executions as simple > criteria for JIT-ing the query. > May be compilation of simple queries (with small cost) should be done only > for prepared statements... > > Another question is whether it is sensible to redundantly do expensive work > (llvm compilation) in all backends. > This question refers to shared prepared statement cache. But even without > such cache, it seems to be possible to use for library name some signature > of the compiled expression and allow > to share this libraries between backends. So before starting code > generation, ExecReadyCompiledExpr can first build signature and check if > correspondent library is already present. > Also it will be easier to control space used by compiled libraries in this > case.
Totally agree; these considerations are very important. I tested several queries in my application that had >30 second compile times against a one second run time,. Not being able to manage when compilation happens is making it difficult to get a sense of llvm performance in the general case. Having explain analyze print compile time and being able to prepare llvm compiled queries ought to help measurement and tuning. There may be utility here beyond large analytical queries as the ability to optimize spreads through the executor with the right trade off management. This work is very exciting...thank you. merlin
