[
https://issues.apache.org/jira/browse/HADOOP-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283422#comment-13283422
]
Harsh J commented on HADOOP-6871:
---------------------------------
{quote}
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.ha.TestHealthMonitor
{quote}
This was perhaps an issue on trunk back then. Passes for me locally:
{code}
➜ hadoop-common svn diff
Index: src/test/java/org/apache/hadoop/conf/TestConfiguration.java
===================================================================
--- src/test/java/org/apache/hadoop/conf/TestConfiguration.java (revision
1342610)
+++ src/test/java/org/apache/hadoop/conf/TestConfiguration.java (working copy)
@@ -999,6 +999,15 @@
"Not returning expected number of classes. Number of returned classes
="
+ classes.length, 0, classes.length);
}
+
+ public void testInvalidSubstitutation() {
+ String key = "test.random.key";
+ String keyExpression = "${" + key + "}";
+ Configuration configuration = new Configuration();
+ configuration.set(key, keyExpression);
+ String value = configuration.get(key);
+ assertTrue("Unexpected value " + value, value.equals(keyExpression));
+ }
public static void main(String[] argv) throws Exception {
junit.textui.TestRunner.main(new String[]{
Index: src/main/java/org/apache/hadoop/conf/Configuration.java
===================================================================
--- src/main/java/org/apache/hadoop/conf/Configuration.java (revision
1342610)
+++ src/main/java/org/apache/hadoop/conf/Configuration.java (working copy)
@@ -617,7 +617,13 @@
}
Matcher match = varPat.matcher("");
String eval = expr;
+ Set<String> evalSet = new HashSet<String>();
for(int s=0; s<MAX_SUBST; s++) {
+ if (evalSet.contains(eval)) {
+ // Cyclic resolution pattern detected. Return current expression.
+ return eval;
+ }
+ evalSet.add(eval);
match.reset(eval);
if (!match.find()) {
return eval;
➜ hadoop-common mvn clean install -Dtest=TestConfiguration,TestHealthMonitor
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Hadoop Common 3.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ hadoop-common ---
[INFO] Deleting
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-common ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/test-dir
[mkdir] Created dir:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/test/data
[INFO] Executed tasks
[INFO]
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @
hadoop-common ---
[INFO] Source directory:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/generated-sources/java
added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.5:add-test-source (add-test-source) @
hadoop-common ---
[INFO] Test Source directory:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java
added.
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (compile-proto) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (save-version) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-dependency-plugin:2.1:build-classpath (build-classpath) @
hadoop-common ---
[INFO] Wrote classpath file
'/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/classes/mrapp-generated-classpath'.
[INFO]
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
hadoop-common ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @
hadoop-common ---
[INFO] Compiling 629 source files to
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/classes
[INFO]
[INFO] --- avro-maven-plugin:1.5.3:schema (generate-avro-test-sources) @
hadoop-common ---
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (compile-test-proto) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (generate-test-sources) @ hadoop-common
---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources) @
hadoop-common ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-log-dir) @ hadoop-common ---
[INFO] Executing tasks
main:
[delete] Deleting directory
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/test/data
[mkdir] Created dir:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/test/data
[mkdir] Created dir:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/log
[copy] Copying 5 files to
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/test-classes
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @
hadoop-common ---
[INFO] Compiling 309 source files to
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ hadoop-common ---
[INFO] Surefire report directory:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.hadoop.conf.TestConfiguration
Tests run: 37, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.672 sec
Running org.apache.hadoop.ha.TestHealthMonitor
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.291 sec
Results :
Tests run: 40, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (prepare-jar) @ hadoop-common ---
[INFO] Building jar:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:test-jar (prepare-test-jar) @ hadoop-common
---
[INFO] Building jar:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT-tests.jar
[INFO]
[INFO] >>> maven-source-plugin:2.1.2:jar (default) @ hadoop-common >>>
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @
hadoop-common ---
[INFO] Source directory:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/generated-sources/java
added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.5:add-test-source (add-test-source) @
hadoop-common ---
[INFO] Test Source directory:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java
added.
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (compile-proto) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (save-version) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-dependency-plugin:2.1:build-classpath (build-classpath) @
hadoop-common ---
[INFO] Skipped writing classpath file
'/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/classes/mrapp-generated-classpath'.
No changes found.
[INFO]
[INFO] <<< maven-source-plugin:2.1.2:jar (default) @ hadoop-common <<<
[INFO]
[INFO] --- maven-source-plugin:2.1.2:jar (default) @ hadoop-common ---
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/hadoop already added, skipping
[INFO] org/apache/hadoop/ha already added, skipping
[INFO] org/apache/hadoop/ipc already added, skipping
[INFO] org/apache/hadoop/ipc/protobuf already added, skipping
[INFO] Building jar:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT-sources.jar
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/hadoop already added, skipping
[INFO] org/apache/hadoop/ha already added, skipping
[INFO] org/apache/hadoop/ipc already added, skipping
[INFO] org/apache/hadoop/ipc/protobuf already added, skipping
[INFO]
[INFO] >>> maven-source-plugin:2.1.2:test-jar (default) @ hadoop-common >>>
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @
hadoop-common ---
[INFO] Source directory:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/generated-sources/java
added.
[INFO]
[INFO] --- build-helper-maven-plugin:1.5:add-test-source (add-test-source) @
hadoop-common ---
[INFO] Test Source directory:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java
added.
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (compile-proto) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (save-version) @ hadoop-common ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-dependency-plugin:2.1:build-classpath (build-classpath) @
hadoop-common ---
[INFO] Skipped writing classpath file
'/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/classes/mrapp-generated-classpath'.
No changes found.
[INFO]
[INFO] <<< maven-source-plugin:2.1.2:test-jar (default) @ hadoop-common <<<
[INFO]
[INFO] --- maven-source-plugin:2.1.2:test-jar (default) @ hadoop-common ---
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/hadoop already added, skipping
[INFO] org/apache/hadoop/io already added, skipping
[INFO] org/apache/hadoop/io/serializer already added, skipping
[INFO] org/apache/hadoop/io/serializer/avro already added, skipping
[INFO] org/apache/hadoop/ipc already added, skipping
[INFO] org/apache/hadoop/ipc/protobuf already added, skipping
[INFO] org/apache/hadoop/record already added, skipping
[INFO] Building jar:
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT-test-sources.jar
[INFO] org already added, skipping
[INFO] org/apache already added, skipping
[INFO] org/apache/hadoop already added, skipping
[INFO] org/apache/hadoop/io already added, skipping
[INFO] org/apache/hadoop/io/serializer already added, skipping
[INFO] org/apache/hadoop/io/serializer/avro already added, skipping
[INFO] org/apache/hadoop/ipc already added, skipping
[INFO] org/apache/hadoop/ipc/protobuf already added, skipping
[INFO] org/apache/hadoop/record already added, skipping
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ hadoop-common ---
[INFO]
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
hadoop-common ---
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ hadoop-common
---
[INFO] Installing
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT.jar
to
/Users/harshchouraria/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-SNAPSHOT/hadoop-common-3.0.0-SNAPSHOT.jar
[INFO] Installing
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/pom.xml
to
/Users/harshchouraria/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-SNAPSHOT/hadoop-common-3.0.0-SNAPSHOT.pom
[INFO] Installing
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT-tests.jar
to
/Users/harshchouraria/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-SNAPSHOT/hadoop-common-3.0.0-SNAPSHOT-tests.jar
[INFO] Installing
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT-sources.jar
to
/Users/harshchouraria/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-SNAPSHOT/hadoop-common-3.0.0-SNAPSHOT-sources.jar
[INFO] Installing
/Users/harshchouraria/Work/code/apache/root-hadoop/trunk/hadoop-common-project/hadoop-common/target/hadoop-common-3.0.0-SNAPSHOT-test-sources.jar
to
/Users/harshchouraria/.m2/repository/org/apache/hadoop/hadoop-common/3.0.0-SNAPSHOT/hadoop-common-3.0.0-SNAPSHOT-test-sources.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.349s
[INFO] Finished at: Fri May 25 19:25:03 GMT+05:30 2012
[INFO] Final Memory: 23M/121M
[INFO] ------------------------------------------------------------------------
{code}
+1, committing.
> When the value of a configuration key is set to its unresolved form, it
> causes the IllegalStateException in Configuration.get() stating that
> substitution depth is too large.
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-6871
> URL: https://issues.apache.org/jira/browse/HADOOP-6871
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Arvind Prabhakar
> Attachments: HADOOP-6871-1.patch, HADOOP-6871-2.patch,
> HADOOP-6871-3.patch, HADOOP-6871.patch
>
>
> When a configuration value is set to its unresolved expression string, it
> leads to recursive substitution attempts in
> {{Configuration.substituteVars(String)}} method until the max substitution
> check kicks in and raises an IllegalStateException indicating that the
> substitution depth is too large. For example, the configuration key
> "{{foobar}}" with a value set to "{{$\{foobar\}}}" will cause this behavior.
> While this is not a usual use case, it can happen in build environments where
> a property value is not specified and yet being passed into the test
> mechanism leading to failures due to this limitation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
