[
https://issues.apache.org/jira/browse/GEODE-9595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirk Lund updated GEODE-9595:
-----------------------------
Description:
I found circular classpath dependencies in the testing targets. This results in
the module such as geode-core adding geode-core*.jar to the classpath of the
tests. Classes and resources are then loaded into the test JVM from the .jar
file instead of from the filesystem. This causes issues only in some tests such
as AnalyzeSerializables integration tests.
1) Reproduce:
a) Delete any line from
geode-core/src/main/resources/org/apache/geode/internal/sanctioned-geode-core-serializables.txt
b) Execute {{$ ./gradlew geode-core:integrationTest --tests
AnalyzeCoreSerializablesIntegrationTest}}
{{testSerializables}} will fail with {{FileSystemNotFoundException}}:
{noformat}
org.apache.geode.codeAnalysis.AnalyzeCoreSerializablesIntegrationTest >
testSerializables FAILED
java.nio.file.FileSystemNotFoundException
at
com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
at
com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
at java.nio.file.Paths.get(Paths.java:143)
at
org.apache.geode.codeAnalysis.AnalyzeDataSerializablesJUnitTestBase.mainResourceToSourcePath(AnalyzeDataSerializablesJUnitTestBase.java:260)
at
org.apache.geode.codeAnalysis.AnalyzeSerializablesJUnitTestBase.testSerializables(AnalyzeSerializablesJUnitTestBase.java:102)
{noformat}
2) Debug:
a) Open {{AnalyzeSerializablesJUnitTestBase}} in IntelliJ and put a breakpoint
on the {{fail}} line at the end of {{testSerializables}}
b) Execute debug AnalyzeCoreSerializablesIntegrationTest in IntelliJ
c) When the debugger hits the breakpoint, add a watch for {{getModuleClass()}}
This will show add an entry to Variables panel of type
{{CoreSanctionedSerializablesService}}
d) Double click on the variable {{CoreSanctionedSerializablesService}} and
navigate to {{classLoader}} | {{ucp}} | {{path}} and note that
{{file:/path/to/geode/geode-core/build/libs/geode-core-1.15.0-build.0.jar}} is
present in the list which means that an integrationTest dependency directly or
indirectly depends on geode-core
e) Without stopping the debugger, navigate to {{mainResourceToSourcePath}} in
{{AnalyzeDataSerializablesJUnitTestBase}} to put a breakpoint on the first line
f) Let the test continue running until the debugger hits the second breakpoint
g) Notice that the parameter {{buildResourceURI}} is pointing to the resource
in the jar file instead of in the filesystem under
{{geode-core/build/resources}}
{noformat}
jar:file:/path/to/geode/geode-core/build/libs/geode-core-1.15.0-build.0.jar!/org/apache/geode/internal/sanctioned-geode-core-serializables.txt
{noformat}
h) Execute {{$ ./gradlew geode-core:dependencies --configuration
integrationTestRuntimeClasspath | less}}
Note that the output is piped to less.
Search for {{geode-core}} and note that the following dependencies will add the
geode-core jar file as a dependency:
{noformat}
+--- project :geode-gfsh
| +--- <snip>
| +--- project :geode-core (*)
{noformat}
{noformat}
+--- project :geode-dunit
| +--- <snip>
| +--- project :geode-core (*)
| +--- <snip>
| +--- project :geode-cq
| | +--- <snip>
| | +--- project :geode-core (*)
| +--- project :geode-log4j
| | +--- <snip>
| | +--- project :geode-core (*)
{noformat}
{noformat}
+--- project :geode-deployment:geode-deployment-legacy
| +--- <snip>
| +--- project :geode-core (*)
{noformat}
3) Fix:
a) Edit {{geode-core/build.gradle}} and retest until the bug no longer
reproduces
In the end, we find that the following changes are required:
{noformat}
- runtimeOnly(project(':geode-deployment:geode-deployment-legacy'))
+ runtimeOnly(project(':geode-deployment:geode-deployment-legacy')) {
+ exclude module: 'geode-core'
+ }
{noformat}
{noformat}
- integrationTestImplementation(project(':geode-gfsh'))
- integrationTestImplementation(project(':geode-junit'))
- integrationTestImplementation(project(':geode-dunit'))
- integrationTestImplementation(project(':geode-log4j'))
+ integrationTestImplementation(project(':geode-gfsh')) {
+ exclude module: 'geode-core'
+ }
+ integrationTestImplementation(project(':geode-junit')) {
+ exclude module: 'geode-core'
+ }
+ integrationTestImplementation(project(':geode-dunit')) {
+ exclude module: 'geode-core'
+ }
+ integrationTestImplementation(project(':geode-log4j')) {
+ exclude module: 'geode-core'
+ }
{noformat}
Since distributedTest also has a circular dependency in its classpath, we also
add the only missing exclusion required there:
{noformat}
- distributedTestImplementation(project(':geode-gfsh'))
+ distributedTestImplementation(project(':geode-gfsh')) {
+ exclude module: 'geode-core'
+ }
{noformat}
b) Failure message of {{AnalyzeCoreSerializablesIntegrationTest}} should now be:
{noformat}
/path/to/geode/geode-core/build/resources/integrationTest/org/apache/geode/codeAnalysis/excludedClasses.txt
Otherwise if this doesn't break backward compatibility, copy the file
/path/to/geode/geode-core/build/integrationTest/test-worker-000000/actualSerializables.dat
to
/path/to/geode/geode-core/src/main/resources/org/apache/geode/internal/sanctioned-geode-core-serializables.txt.
at
org.apache.geode.codeAnalysis.AnalyzeSerializablesJUnitTestBase.testSerializables(AnalyzeSerializablesJUnitTestBase.java:100)
{noformat}
...instead of the {{java.nio.file.FileSystemNotFoundException}}
was:
I found circular dependencies that cause failures in new tests in geode-core,
geode-cq, geode-gfsh, geode-management, geode-wan, geode-web-api.
The circular dependencies result in the module such as geode-core adding
geode-core*.jar to the classpath of the tests. Classes and resources are then
loaded into the test JVM from the .jar file instead of from the filesystem.
This causes issues only in some tests such as AnalyzeSerializables integration
tests.
1) Reproduce:
a) Delete any line from
geode-core/src/main/resources/org/apache/geode/internal/sanctioned-geode-core-serializables.txt
b) Execute {{$ ./gradlew geode-core:integrationTest --tests
AnalyzeCoreSerializablesIntegrationTest}}
{{testSerializables}} will fail with {{FileSystemNotFoundException}}:
{noformat}
org.apache.geode.codeAnalysis.AnalyzeCoreSerializablesIntegrationTest >
testSerializables FAILED
java.nio.file.FileSystemNotFoundException
at
com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
at
com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
at java.nio.file.Paths.get(Paths.java:143)
at
org.apache.geode.codeAnalysis.AnalyzeDataSerializablesJUnitTestBase.mainResourceToSourcePath(AnalyzeDataSerializablesJUnitTestBase.java:260)
at
org.apache.geode.codeAnalysis.AnalyzeSerializablesJUnitTestBase.testSerializables(AnalyzeSerializablesJUnitTestBase.java:102)
{noformat}
2) Debug:
a) Open {{AnalyzeSerializablesJUnitTestBase}} in IntelliJ and put a breakpoint
on the {{fail}} line at the end of {{testSerializables}}
b) Execute debug AnalyzeCoreSerializablesIntegrationTest in IntelliJ
c) When the debugger hits the breakpoint, add a watch for {{getModuleClass()}}
This will show add an entry to Variables panel of type
{{CoreSanctionedSerializablesService}}
d) Double click on the variable {{CoreSanctionedSerializablesService}} and
navigate to {{classLoader}} | {{ucp}} | {{path}} and note that
{{file:/path/to/geode/geode-core/build/libs/geode-core-1.15.0-build.0.jar}} is
present in the list which means that an integrationTest dependency directly or
indirectly depends on geode-core
e) Without stopping the debugger, navigate to {{mainResourceToSourcePath}} in
{{AnalyzeDataSerializablesJUnitTestBase}} to put a breakpoint on the first line
f) Let the test continue running until the debugger hits the second breakpoint
g) Notice that the parameter {{buildResourceURI}} is pointing to the resource
in the jar file instead of in the filesystem under
{{geode-core/build/resources}}
{noformat}
jar:file:/path/to/geode/geode-core/build/libs/geode-core-1.15.0-build.0.jar!/org/apache/geode/internal/sanctioned-geode-core-serializables.txt
{noformat}
h) Execute {{$ ./gradlew geode-core:dependencies --configuration
integrationTestRuntimeClasspath | less}}
Note that the output is piped to less.
Search for {{geode-core}} and note that the following dependencies will add the
geode-core jar file as a dependency:
{noformat}
+--- project :geode-gfsh
| +--- <snip>
| +--- project :geode-core (*)
{noformat}
{noformat}
+--- project :geode-dunit
| +--- <snip>
| +--- project :geode-core (*)
| +--- <snip>
| +--- project :geode-cq
| | +--- <snip>
| | +--- project :geode-core (*)
| +--- project :geode-log4j
| | +--- <snip>
| | +--- project :geode-core (*)
{noformat}
{noformat}
+--- project :geode-deployment:geode-deployment-legacy
| +--- <snip>
| +--- project :geode-core (*)
{noformat}
3) Fix:
a) Edit {{geode-core/build.gradle}} and retest until the bug no longer
reproduces
In the end, we find that the following changes are required:
{noformat}
- runtimeOnly(project(':geode-deployment:geode-deployment-legacy'))
+ runtimeOnly(project(':geode-deployment:geode-deployment-legacy')) {
+ exclude module: 'geode-core'
+ }
{noformat}
{noformat}
- integrationTestImplementation(project(':geode-gfsh'))
- integrationTestImplementation(project(':geode-junit'))
- integrationTestImplementation(project(':geode-dunit'))
- integrationTestImplementation(project(':geode-log4j'))
+ integrationTestImplementation(project(':geode-gfsh')) {
+ exclude module: 'geode-core'
+ }
+ integrationTestImplementation(project(':geode-junit')) {
+ exclude module: 'geode-core'
+ }
+ integrationTestImplementation(project(':geode-dunit')) {
+ exclude module: 'geode-core'
+ }
+ integrationTestImplementation(project(':geode-log4j')) {
+ exclude module: 'geode-core'
+ }
{noformat}
Since distributedTest also has a circular dependency in its classpath, we also
add the only missing exclusion required there:
{noformat}
- distributedTestImplementation(project(':geode-gfsh'))
+ distributedTestImplementation(project(':geode-gfsh')) {
+ exclude module: 'geode-core'
+ }
{noformat}
b) Failure message of {{AnalyzeCoreSerializablesIntegrationTest}} should now be:
{noformat}
/path/to/geode/geode-core/build/resources/integrationTest/org/apache/geode/codeAnalysis/excludedClasses.txt
Otherwise if this doesn't break backward compatibility, copy the file
/path/to/geode/geode-core/build/integrationTest/test-worker-000000/actualSerializables.dat
to
/path/to/geode/geode-core/src/main/resources/org/apache/geode/internal/sanctioned-geode-core-serializables.txt.
at
org.apache.geode.codeAnalysis.AnalyzeSerializablesJUnitTestBase.testSerializables(AnalyzeSerializablesJUnitTestBase.java:100)
{noformat}
...instead of the {{java.nio.file.FileSystemNotFoundException}}
> Analyze serializables testSerializables fails with FileSystemNotFoundException
> ------------------------------------------------------------------------------
>
> Key: GEODE-9595
> URL: https://issues.apache.org/jira/browse/GEODE-9595
> Project: Geode
> Issue Type: Bug
> Components: build
> Reporter: Kirk Lund
> Assignee: Kirk Lund
> Priority: Major
> Labels: pull-request-available
>
> I found circular classpath dependencies in the testing targets. This results
> in the module such as geode-core adding geode-core*.jar to the classpath of
> the tests. Classes and resources are then loaded into the test JVM from the
> .jar file instead of from the filesystem. This causes issues only in some
> tests such as AnalyzeSerializables integration tests.
> 1) Reproduce:
> a) Delete any line from
> geode-core/src/main/resources/org/apache/geode/internal/sanctioned-geode-core-serializables.txt
> b) Execute {{$ ./gradlew geode-core:integrationTest --tests
> AnalyzeCoreSerializablesIntegrationTest}}
> {{testSerializables}} will fail with {{FileSystemNotFoundException}}:
> {noformat}
> org.apache.geode.codeAnalysis.AnalyzeCoreSerializablesIntegrationTest >
> testSerializables FAILED
> java.nio.file.FileSystemNotFoundException
> at
> com.sun.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:171)
> at
> com.sun.nio.zipfs.ZipFileSystemProvider.getPath(ZipFileSystemProvider.java:157)
> at java.nio.file.Paths.get(Paths.java:143)
> at
> org.apache.geode.codeAnalysis.AnalyzeDataSerializablesJUnitTestBase.mainResourceToSourcePath(AnalyzeDataSerializablesJUnitTestBase.java:260)
> at
> org.apache.geode.codeAnalysis.AnalyzeSerializablesJUnitTestBase.testSerializables(AnalyzeSerializablesJUnitTestBase.java:102)
> {noformat}
> 2) Debug:
> a) Open {{AnalyzeSerializablesJUnitTestBase}} in IntelliJ and put a
> breakpoint on the {{fail}} line at the end of {{testSerializables}}
> b) Execute debug AnalyzeCoreSerializablesIntegrationTest in IntelliJ
> c) When the debugger hits the breakpoint, add a watch for {{getModuleClass()}}
> This will show add an entry to Variables panel of type
> {{CoreSanctionedSerializablesService}}
> d) Double click on the variable {{CoreSanctionedSerializablesService}} and
> navigate to {{classLoader}} | {{ucp}} | {{path}} and note that
> {{file:/path/to/geode/geode-core/build/libs/geode-core-1.15.0-build.0.jar}}
> is present in the list which means that an integrationTest dependency
> directly or indirectly depends on geode-core
> e) Without stopping the debugger, navigate to {{mainResourceToSourcePath}} in
> {{AnalyzeDataSerializablesJUnitTestBase}} to put a breakpoint on the first
> line
> f) Let the test continue running until the debugger hits the second breakpoint
> g) Notice that the parameter {{buildResourceURI}} is pointing to the resource
> in the jar file instead of in the filesystem under
> {{geode-core/build/resources}}
> {noformat}
> jar:file:/path/to/geode/geode-core/build/libs/geode-core-1.15.0-build.0.jar!/org/apache/geode/internal/sanctioned-geode-core-serializables.txt
> {noformat}
> h) Execute {{$ ./gradlew geode-core:dependencies --configuration
> integrationTestRuntimeClasspath | less}}
> Note that the output is piped to less.
> Search for {{geode-core}} and note that the following dependencies will add
> the geode-core jar file as a dependency:
> {noformat}
> +--- project :geode-gfsh
> | +--- <snip>
> | +--- project :geode-core (*)
> {noformat}
> {noformat}
> +--- project :geode-dunit
> | +--- <snip>
> | +--- project :geode-core (*)
> | +--- <snip>
> | +--- project :geode-cq
> | | +--- <snip>
> | | +--- project :geode-core (*)
> | +--- project :geode-log4j
> | | +--- <snip>
> | | +--- project :geode-core (*)
> {noformat}
> {noformat}
> +--- project :geode-deployment:geode-deployment-legacy
> | +--- <snip>
> | +--- project :geode-core (*)
> {noformat}
> 3) Fix:
> a) Edit {{geode-core/build.gradle}} and retest until the bug no longer
> reproduces
> In the end, we find that the following changes are required:
> {noformat}
> - runtimeOnly(project(':geode-deployment:geode-deployment-legacy'))
> + runtimeOnly(project(':geode-deployment:geode-deployment-legacy')) {
> + exclude module: 'geode-core'
> + }
> {noformat}
> {noformat}
> - integrationTestImplementation(project(':geode-gfsh'))
> - integrationTestImplementation(project(':geode-junit'))
> - integrationTestImplementation(project(':geode-dunit'))
> - integrationTestImplementation(project(':geode-log4j'))
> + integrationTestImplementation(project(':geode-gfsh')) {
> + exclude module: 'geode-core'
> + }
> + integrationTestImplementation(project(':geode-junit')) {
> + exclude module: 'geode-core'
> + }
> + integrationTestImplementation(project(':geode-dunit')) {
> + exclude module: 'geode-core'
> + }
> + integrationTestImplementation(project(':geode-log4j')) {
> + exclude module: 'geode-core'
> + }
> {noformat}
> Since distributedTest also has a circular dependency in its classpath, we
> also add the only missing exclusion required there:
> {noformat}
> - distributedTestImplementation(project(':geode-gfsh'))
> + distributedTestImplementation(project(':geode-gfsh')) {
> + exclude module: 'geode-core'
> + }
> {noformat}
> b) Failure message of {{AnalyzeCoreSerializablesIntegrationTest}} should now
> be:
> {noformat}
> /path/to/geode/geode-core/build/resources/integrationTest/org/apache/geode/codeAnalysis/excludedClasses.txt
> Otherwise if this doesn't break backward compatibility, copy the file
> /path/to/geode/geode-core/build/integrationTest/test-worker-000000/actualSerializables.dat
> to
> /path/to/geode/geode-core/src/main/resources/org/apache/geode/internal/sanctioned-geode-core-serializables.txt.
> at
> org.apache.geode.codeAnalysis.AnalyzeSerializablesJUnitTestBase.testSerializables(AnalyzeSerializablesJUnitTestBase.java:100)
> {noformat}
> ...instead of the {{java.nio.file.FileSystemNotFoundException}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
