[
https://issues.apache.org/jira/browse/IMPALA-9153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-9153:
----------------------------------
Component/s: Infrastructure
> Avoid unneccessary cup compilation
> ----------------------------------
>
> Key: IMPALA-9153
> URL: https://issues.apache.org/jira/browse/IMPALA-9153
> Project: IMPALA
> Issue Type: Test
> Components: Infrastructure
> Reporter: Quanlong Huang
> Priority: Major
>
> Everytime when I run a single FE test, maven always recompile the whole
> module since it detects changes. The change is made by cup-maven-plugin. It
> recompiles sql-parser.cup to SqlParser.java and SqlParserSymbols.java
> everytime even there're no changes in sql-parser.cup. The recompilation cost
> some time and slow down the reproducing of some flasky test that requires
> running it many times.
> An output example:
> {code:java}
> $ (pushd fe && mvn test -fae -Dtest=AuthorizationStmtTest#test*Enabled[1])
> ~/workspace/Impala/fe ~/workspace/Impala
> [INFO] Scanning for projects...
> [INFO]
> [INFO] -----------------< org.apache.impala:impala-frontend
> >------------------
> [INFO] Building Apache Impala Query Engine Frontend 0.1-SNAPSHOT
> [INFO] --------------------------------[ jar
> ]---------------------------------
> [WARNING] Could not transfer metadata
> com.cloudera.cdh:cdh-root:6.x-SNAPSHOT/maven-metadata.xml from/to
> ${distMgmtSnapshotsId} (${distMgmtSnapshotsUrl}): Cannot access
> ${distMgmtSnapshotsUrl} with type default using the available connector
> factories: BasicRepositoryConnectorFactory
> [INFO]
> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce
> (enforce-banned-dependencies) @ impala-frontend ---
> [INFO]
> [INFO] --- jacoco-maven-plugin:0.7.6.201602180812:prepare-agent
> (prepare-jacoco-agent) @ impala-frontend ---
> [INFO] Skipping JaCoCo execution because property jacoco.skip is set.
> [INFO] surefireJacocoArg set to empty
> [INFO]
> [INFO] --- cup-maven-plugin:1.6-cdh:generate (cup) @ impala-frontend ---
> [INFO] CUP: Processing 1 cup files <------------ Always regenerate the files
> Warning : Terminal "EMPTY_IDENT" was declared but never used
> Warning : Terminal "UNEXPECTED_CHAR" was declared but never used
> ------- CUP v0.11a czt01 beta Parser Generation Summary -------
> 0 errors and 2 warnings
> 229 terminals, 228 non-terminals, and 875 productions declared,
> producing 1620 unique parse states.
> 2 terminals declared but not used.
> 0 non-terminals declared but not used.
> 0 productions never reduced.
> 0 conflicts detected (0 expected).
> Code written to "SqlParser.java", and "SqlParserSymbols.java".
> ---------------------------------------------------- (v0.11a czt01 beta)
> [WARNING] /home/quanlong/workspace/Impala/fe/src/main/cup/sql-parser.cup
> [0:0]: Terminal "EMPTY_IDENT" was declared but never used
> [WARNING] /home/quanlong/workspace/Impala/fe/src/main/cup/sql-parser.cup
> [0:0]: Terminal "UNEXPECTED_CHAR" was declared but never used
> [INFO] CUP: generated
> /home/quanlong/workspace/Impala/fe/target/generated-sources/cup/org/apache/impala/analysis/SqlParser.java
> [INFO] CUP: generated
> /home/quanlong/workspace/Impala/fe/target/generated-sources/cup/org/apache/impala/analysis/SqlParserSymbols.java
> [INFO] CUP: generated
> /home/quanlong/workspace/Impala/fe/target/generated-sources/cup/org/apache/impala/analysis/action_table.dat
> [INFO]
> [INFO] --- maven-jflex-plugin:1.4.3:generate (jflex) @ impala-frontend ---
> [INFO] SqlScanner.java is up to date.
> [INFO]
> [INFO] --- maven-dependency-plugin:3.1.1:build-classpath (write-classpath) @
> impala-frontend ---
> [INFO] Skipped writing classpath file
> '/home/quanlong/workspace/Impala/fe/target/build-classpath.txt'. No changes
> found.
> [INFO]
> [INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @
> impala-frontend ---
> [INFO] Source directory:
> /home/quanlong/workspace/Impala/fe/generated-sources/gen-java added.
> [INFO] Source directory:
> /home/quanlong/workspace/Impala/fe/target/generated-sources/cup added.
> [INFO] Source directory:
> /home/quanlong/workspace/Impala/fe/src/compat-hive-2/java added.
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
> impala-frontend ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory
> /home/quanlong/workspace/Impala/fe/src/main/resources
> [INFO] Copying 1 resource
> [INFO]
> [INFO] --- maven-compiler-plugin:3.3:compile (default-compile) @
> impala-frontend ---
> [INFO] Changes detected - recompiling the module! <--------- Decide to
> recomple
> [INFO] Compiling 907 source files to
> /home/quanlong/workspace/Impala/fe/target/classes
> [WARNING]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[24,16]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[24,16]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[24,16]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[33,23]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[40,15]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [WARNING]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/util/UnsafeUtil.java:[45,25]
> sun.misc.Unsafe is internal proprietary API and may be removed in a future
> release
> [INFO]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java:
> Some input files use or override a deprecated API.
> [INFO]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/hive/executor/UdfExecutor.java:
> Recompile with -Xlint:deprecation for details.
> [INFO]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/analysis/Expr.java:
> Some input files use unchecked or unsafe operations.
> [INFO]
> /home/quanlong/workspace/Impala/fe/src/main/java/org/apache/impala/analysis/Expr.java:
> Recompile with -Xlint:unchecked for details.
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @
> impala-frontend ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 43 resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.3:testCompile (default-testCompile) @
> impala-frontend ---
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-surefire-plugin:2.20:test (default-test) @ impala-frontend
> ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO] T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running org.apache.impala.authorization.AuthorizationStmtTest
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 69.847
> s - in org.apache.impala.authorization.AuthorizationStmtTest
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
> [INFO] --- jacoco-maven-plugin:0.7.6.201602180812:report (jacoco-report) @
> impala-frontend ---
> [INFO] Skipping JaCoCo execution because property jacoco.skip is set.
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 01:23 min
> [INFO] Finished at: 2019-11-13T15:50:33+08:00
> [INFO]
> ------------------------------------------------------------------------
> {code}
> We should configure cup-maven-plugin to avoid unneccessary recompilation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
