[
https://issues.apache.org/jira/browse/DRILL-5052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15762420#comment-15762420
]
ASF GitHub Bot commented on DRILL-5052:
---------------------------------------
Github user jinfengni commented on a diff in the pull request:
https://github.com/apache/drill/pull/660#discussion_r93122823
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/compile/ClassBuilder.java ---
@@ -0,0 +1,189 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.compile;
+
+import java.io.File;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.util.Map;
+
+import org.apache.drill.common.config.DrillConfig;
+import org.apache.drill.exec.compile.ClassTransformer.ClassNames;
+import org.apache.drill.exec.exception.ClassTransformationException;
+import org.apache.drill.exec.expr.CodeGenerator;
+import org.apache.drill.exec.server.options.OptionManager;
+import org.codehaus.commons.compiler.CompileException;
+
+/**
+ * Implements the "plain-old Java" method of code generation and
+ * compilation. Given a {@link CodeGenerator}, obtains the generated
+ * source code, compiles it with the selected compiler, loads the
+ * byte-codes into a class loader and provides the resulting
+ * class. Compared with the {@link ClassTransformer} mechanism,
+ * this one requires the code generator to have generated a complete
+ * Java class that is capable of direct compilation and loading.
+ * This means the generated class must be a subclass of the template
+ * so that the JVM can use normal Java inheritance to associate the
+ * template and generated methods.
+ * <p>
+ * Here is how to use the plain-old Java technique to debug
+ * generated code:
+ * <ul>
+ * <li>Set the config option <var>drill.exec.compile.save_source</var>
+ * to <var>true</var>.</li>
+ * <li>Set the config option <var>drill.exec.compile.code_dir</var>
+ * to the location where you want to save the generated source
+ * code.</li>
+ * <li>Where you generate code (using a {@link CodeGenerator}),
+ * set the "plain-old Java" options:<pre>
+ * CodeGenerator<Foo> cg = ...
+ * cg.plainOldJavaCapable(true); // Class supports plain-old Java
+ * cg.preferPlainOldJava(true); // Actually generate plain-old Java
+ * ...</pre></li>
+ * <li>In your favorite IDE, add to the code lookup path the
+ * code directory saved earlier. In Eclipse, for example, you do
+ * this in the debug configuration you will use to debug Drill.</li>
+ * <li>Set a breakpoint in template used for the generated code.</li>
+ * <li>Run Drill. The IDE will stop at your breakpoint.</li>
+ * <li>Step into the generated code. Examine class field and
+ * local variables. Have fun!</li>
+ * </ul>
+ * <p>
+ * Note: not all generated code is ready to be compiled as plain-old
+ * Java. Some classes omit from the template the proper <code>throws</code>
+ * declarations. Other minor problems may also crop up. All are easy
+ * to fix. Once you've done so, add the following to mark that you've
+ * done the clean-up:<pre>
+ * cg.plainOldJavaCapable(true); // Class supports plain-old Java</pre>
+ * <p>
+ * The setting to prefer plain-old Java is ignored for generated
+ * classes not marked as plain-old Java capable.
+ */
+
+public class ClassBuilder {
+
+ public static final String SAVE_CODE_OPTION = CodeCompiler.COMPILE_BASE
+ ".save_source";
+ public static final String CODE_DIR_OPTION = CodeCompiler.COMPILE_BASE +
".code_dir";
+
+ private final DrillConfig config;
+ private final OptionManager options;
+ private final boolean saveCode;
+ private final File codeDir;
+
+ public ClassBuilder(DrillConfig config, OptionManager optionManager) {
+ this.config = config;
+ options = optionManager;
+
+ // The option to save code is a boot-time option because
+ // it is used selectively during debugging, but can cause
+ // excessive I/O in a running server if used to save all code.
+
+ saveCode = config.getBoolean(SAVE_CODE_OPTION);
+ codeDir = new File(config.getString(CODE_DIR_OPTION));
+ }
+
+ /**
+ * Given a code generator which has already generated plain-old Java
+ * code, compile the code, create a class loader, and return the
+ * resulting Java class.
+ *
+ * @param cg a plain-old Java capable code generator that has generated
+ * plain-old Java code
+ * @return the class that the code generator defines
+ * @throws ClassTransformationException
+ */
+
+ public Class<?> getImplementationClass(CodeGenerator<?> cg) throws
ClassTransformationException {
+ try {
+ return compileClass(cg);
+ } catch (CompileException | ClassNotFoundException|IOException e) {
+ throw new ClassTransformationException(e);
+ }
+ }
+
+ /**
+ * Performs the actual work of compiling the code and loading the class.
+ *
+ * @param cg the code generator that has built the class(es) to be
generated.
+ * @return the class, after code generation and (if needed) compilation.
+ * @throws IOException if an error occurs when optionally writing code
to disk.
+ * @throws CompileException if the generated code has compile issues.
+ * @throws ClassNotFoundException if the generated code references
unknown classes.
+ * @throws ClassTransformationException generic "something is wrong"
error from
+ * Drill class compilation code.
+ */
+ private Class<?> compileClass(CodeGenerator<?> cg) throws IOException,
CompileException, ClassNotFoundException, ClassTransformationException {
+
+ // Get the plain-old Java code.
+
+ String code = cg.getGeneratedCode();
+
+ // Get the class names (dotted, file path, etc.)
+
+ String className = cg.getMaterializedClassName();
+ ClassTransformer.ClassNames name = new
ClassTransformer.ClassNames(className);
+
+ // A key advantage of this method is that the code can be
+ // saved and debugged, if needed.
+
+ saveCode(code, name);
+
+ // Compile the code and load it into a class loader.
+
+ CachedClassLoader classLoader = new CachedClassLoader();
--- End diff --
Why do we created a new CachedClassLoader per compileClass() call? Does it
mean that we always have cache miss, and end up with re-compile the code?
> Option to debug generated Java code using an IDE
> ------------------------------------------------
>
> Key: DRILL-5052
> URL: https://issues.apache.org/jira/browse/DRILL-5052
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Codegen
> Affects Versions: 1.8.0
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Minor
> Labels: ready-to-commit
>
> Drill makes extensive use of Java code generation to implement its operators.
> Drill uses sophisticated techniques to blend generated code with pre-compiled
> template code. An unfortunate side-effect of this behavior is that it is very
> difficult to visualize and debug the generated code.
> As it turns out, Drill's code-merge facility is, in essence, a do-it-yourself
> version of subclassing. The Drill "template" is the parent class, the
> generated code is the subclass. But, rather than using plain-old subclassing,
> Drill combines the code from the two classes into a single "artificial"
> packet of byte codes for which no source exists.
> Modify the code generation path to optionally allow "plain-old Java"
> compilation: the generated code is a subclass of the template. Compile the
> generated code as a plain-old Java class with no byte-code fix-up. Write the
> code to a known location that the IDE can search when looking for source
> files.
> With this change, developers can turn on the above feature, set a breakpoint
> in a template, then step directly into the generated Java code called from
> the template.
> This feature should be an option, enabled by developers when needed. The
> existing byte-code technique should be used for production code generation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
