[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361837#comment-17361837
]
Michael Semb Wever commented on CASSANDRA-16704:
Testing both patches.
Between builds on cassandra-4.0.0 and these patches, the {{lib/}} and
{{build/lib/jars/}} result in identical contents.
But the {{build/test/lib/jars/}} directory contains differences:
{code:java}
❯ diff <(ls build/test/lib/jars/) <(ls ../cassandra/build/test/lib/jars/)
11d10
< assertj-core-3.15.0.jar
15,18d13
< byteman-4.0.6.jar
< byteman-bmunit-4.0.6.jar
< byteman-install-4.0.6.jar
< byteman-submit-4.0.6.jar
22d16
< compile-command-annotations-1.2.0.jar {code}
;that is those jar files no longer appear under {{build/test/lib/jars/}} which
is the intentional of the patch (as they are already "provided" and so under
{{build/lib/jars/}}).
CI:
!https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/862/badge/icon|id=badgeUrl!
> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
> Issue Type: Bug
> Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: cleanup-imports.patch, dedup-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars.
> The tests would be more reflective of the behavior of a runtime environment
> if the test classpath only contained jars that are bundled with the binary
> release, together with explicit test dependencies. Ideally we'd use the
> build/lib/ jars for the classpath since that's what gets packaged, but since
> these aren't available at test compile time and should be identical to lib/
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to
> "org.apache.commons.lang", which is not available at runtime (should be
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both
> test/ and src/ classes, and makes some simple substitutions in the tests such
> as using AbstractMap.SimpleEntry in place of
> org.apache.commons.collections.keyvalue.AbstractMapEntry.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17360242#comment-17360242 ] Angelo Polo commented on CASSANDRA-16704: - The patch 'cleanup-imports.patch' is simply the old patch with the change to build.xml removed. The patch 'dedup-deps.patch' contains the following changes, exclusively modifying build.xml, from my understanding of our discussion above: * Removed duplicates of test scope that are already in provided scope. * Removed duplicated scope attributes. All scope attributes are now uniformly provided in parent-pom. * Added to parent-pom. Was previously only declared in build-deps-pom. > Fix imports; run tests with packaged dependencies > - > > Key: CASSANDRA-16704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16704 > Project: Cassandra > Issue Type: Bug > Components: Build, Test/burn, Test/unit >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Fix For: 4.0.x, 4.x > > Attachments: cleanup-imports.patch, dedup-deps.patch > > > Tests are currently run with a classpath containing _all_ downloaded jars. > The tests would be more reflective of the behavior of a runtime environment > if the test classpath only contained jars that are bundled with the binary > release, together with explicit test dependencies. Ideally we'd use the > build/lib/ jars for the classpath since that's what gets packaged, but since > these aren't available at test compile time and should be identical to lib/ > anyway, I've used the later. > Doing so exposed a couple of references in src/java to > "org.apache.commons.lang", which is not available at runtime (should be > "org.apache.commons.lang*3*"). > Attached patch modifies the test classpath, fixes various imports in both > test/ and src/ classes, and makes some simple substitutions in the tests such > as using AbstractMap.SimpleEntry in place of > org.apache.commons.collections.keyvalue.AbstractMapEntry. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359317#comment-17359317 ] Michael Semb Wever commented on CASSANDRA-16704: bq. Is it intentional that the scope for some entries is repeated across the two pom definitions, while missing for others? Not in general. Definitely jsr305 and compile-command-annotations looks like they can have the duplicated scope removed. I think the others are just explicit because they are overridden, but yes: duplicated overrides are a bit silly. byteman-* for example could be defined as provided, and then overridden to test in build-deps-pom. I can't remember how optional interferes with the scope (ref hadoop* deps). > Fix imports; run tests with packaged dependencies > - > > Key: CASSANDRA-16704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16704 > Project: Cassandra > Issue Type: Bug > Components: Build, Test/burn, Test/unit >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Fix For: 4.0.x, 4.x > > Attachments: test-with-runtime-deps.patch > > > Tests are currently run with a classpath containing _all_ downloaded jars. > The tests would be more reflective of the behavior of a runtime environment > if the test classpath only contained jars that are bundled with the binary > release, together with explicit test dependencies. Ideally we'd use the > build/lib/ jars for the classpath since that's what gets packaged, but since > these aren't available at test compile time and should be identical to lib/ > anyway, I've used the later. > Doing so exposed a couple of references in src/java to > "org.apache.commons.lang", which is not available at runtime (should be > "org.apache.commons.lang*3*"). > Attached patch modifies the test classpath, fixes various imports in both > test/ and src/ classes, and makes some simple substitutions in the tests such > as using AbstractMap.SimpleEntry in place of > org.apache.commons.collections.keyvalue.AbstractMapEntry. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359125#comment-17359125
]
Angelo Polo commented on CASSANDRA-16704:
-
{quote}They are in places, see all those marked as "provided" or "optional"
[here|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/build.xml#L491-L657]
and
[here|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/build.xml#L749-L833].
{quote}
Is it intentional that the scope for some entries is repeated across the two
pom definitions, while missing for others? Below is for 'provided' scope,
similar situation for 'test' scope. Those shown in blue are declared with scope
in only one place, those in green are given a scope in both places:
parent-pom:
{color:#4c9aff}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#4c9aff}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
all-pom:
{color:#4c9aff}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}
> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
> Issue Type: Bug
> Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: test-with-runtime-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars.
> The tests would be more reflective of the behavior of a runtime environment
> if the test classpath only contained jars that are bundled with the binary
> release, together with explicit test dependencies. Ideally we'd use the
> build/lib/ jars for the classpath since that's what gets packaged, but since
> these aren't available at test compile time and should be identical to lib/
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to
> "org.apache.commons.lang", which is not available at runtime (should be
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both
> test/ and src/ classes, and makes some simple substitutions in the tests such
> as using AbstractMap.SimpleEntry in place of
> org.apache.commons.collections.keyvalue.AbstractMapEntry.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1735#comment-1735 ] Michael Semb Wever commented on CASSANDRA-16704: A second patch would be good. And I agree with your analysis of TestRateLimiter.java, something seems off there, though maybe it's how byteman (mutation_limiter.btm) needs its classpaths setup…? > Fix imports; run tests with packaged dependencies > - > > Key: CASSANDRA-16704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16704 > Project: Cassandra > Issue Type: Bug > Components: Build, Test/burn, Test/unit >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Fix For: 4.0.x, 4.x > > Attachments: test-with-runtime-deps.patch > > > Tests are currently run with a classpath containing _all_ downloaded jars. > The tests would be more reflective of the behavior of a runtime environment > if the test classpath only contained jars that are bundled with the binary > release, together with explicit test dependencies. Ideally we'd use the > build/lib/ jars for the classpath since that's what gets packaged, but since > these aren't available at test compile time and should be identical to lib/ > anyway, I've used the later. > Doing so exposed a couple of references in src/java to > "org.apache.commons.lang", which is not available at runtime (should be > "org.apache.commons.lang*3*"). > Attached patch modifies the test classpath, fixes various imports in both > test/ and src/ classes, and makes some simple substitutions in the tests such > as using AbstractMap.SimpleEntry in place of > org.apache.commons.collections.keyvalue.AbstractMapEntry. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356637#comment-17356637
]
Angelo Polo commented on CASSANDRA-16704:
-
I would think so. I can check and create a second patch for those.
A side comment:
{quote}Maybe there are jar files here that should be removed, i.e. should not
even be part the provided scope?
{quote}
Byteman in particular seems to be in 'provided' because of the single class
src/java/org/apache/cassandra/utils/TestRateLimiter.java, which appears to be
only used for testing. Without knowing the details of how byteman works, since
this class doesn't seem to make any reference to other Cassandra classes, the
package name would be the only relevant thing here (if even that) and it could
be moved to test/. The byteman deps could then all be 'test' scoped only. Or is
this class in src/ simply because it could be used by test/unit/,
test/distributed/, etc., whose trees shouldn't depend on each other? (That
would seem a weak reason to include a class with ordinarily unresolvable
imports in the end product, but so long as no one uses it no harm done, and I'm
not up to speed on the organization of tests categories anyhow.)
> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
> Issue Type: Bug
> Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: test-with-runtime-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars.
> The tests would be more reflective of the behavior of a runtime environment
> if the test classpath only contained jars that are bundled with the binary
> release, together with explicit test dependencies. Ideally we'd use the
> build/lib/ jars for the classpath since that's what gets packaged, but since
> these aren't available at test compile time and should be identical to lib/
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to
> "org.apache.commons.lang", which is not available at runtime (should be
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both
> test/ and src/ classes, and makes some simple substitutions in the tests such
> as using AbstractMap.SimpleEntry in place of
> org.apache.commons.collections.keyvalue.AbstractMapEntry.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356586#comment-17356586 ] Michael Semb Wever commented on CASSANDRA-16704: bq. Though in terms of classpath, we're effectively back where we started since instead of joining the compile and provided scopes from build/lib/ with the test scope in build/test/lib/ we'd be joining the compile scope in lib/ with the test and provided scopes in build/test/lib/. Yeah, we have three separate concerns: - compiling, requires compile and provided scopes, - packaging, requires compile scope, - testing, requires compile and test scopes, and sometimes provided/optional scope. I'm open to suggestions on how to tackle this, but not sure the current patch to build.xml really puts us in a significantly better situation. The rest of the patch LGTM! bq. there's no need to duplicate dependencies across provided and test. Shouldn't we be able to remove the test scoped duplicated lines then? > Fix imports; run tests with packaged dependencies > - > > Key: CASSANDRA-16704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16704 > Project: Cassandra > Issue Type: Bug > Components: Build, Test/burn, Test/unit >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Fix For: 4.0.x, 4.x > > Attachments: test-with-runtime-deps.patch > > > Tests are currently run with a classpath containing _all_ downloaded jars. > The tests would be more reflective of the behavior of a runtime environment > if the test classpath only contained jars that are bundled with the binary > release, together with explicit test dependencies. Ideally we'd use the > build/lib/ jars for the classpath since that's what gets packaged, but since > these aren't available at test compile time and should be identical to lib/ > anyway, I've used the later. > Doing so exposed a couple of references in src/java to > "org.apache.commons.lang", which is not available at runtime (should be > "org.apache.commons.lang*3*"). > Attached patch modifies the test classpath, fixes various imports in both > test/ and src/ classes, and makes some simple substitutions in the tests such > as using AbstractMap.SimpleEntry in place of > org.apache.commons.collections.keyvalue.AbstractMapEntry. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356571#comment-17356571 ] Angelo Polo commented on CASSANDRA-16704: - If the 'provided' scope is intentionally part of the test dependencies (and not just the 'test' scope), then does it make sense to remove the exclusion of 'provided' from [test jars resolution|https://github.com/apache/cassandra/blob/3282f5ecf187ecbb56b8d73ab9a9110c010898b0/.build/build-resolver.xml#L178]? Unless there's some other detail with the transitive dependencies, looks like this is what provided is for: "A dependency with this scope is added to the classpath used for compilation and test, but not the runtime classpath." ([http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope).] Then as a bonus, there's no need to duplicate dependencies across provided and test. For example: ./build.xml:733: ./build.xml:734: ./build.xml:735: ./build.xml:736: ./build.xml:830: ./build.xml:831: ./build.xml:832: ./build.xml:833: Though in terms of classpath, we're effectively back where we started since instead of joining the compile and provided scopes from build/lib/ with the test scope in build/test/lib/ we'd be joining the compile scope in lib/ with the test and provided scopes in build/test/lib/. So let me know if you me to remove the modification to build.xml from the patch. At a minimum, I think the two changes to src/ are required for runtime correctness. > Fix imports; run tests with packaged dependencies > - > > Key: CASSANDRA-16704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16704 > Project: Cassandra > Issue Type: Bug > Components: Build, Test/burn, Test/unit >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Fix For: 4.0.x, 4.x > > Attachments: test-with-runtime-deps.patch > > > Tests are currently run with a classpath containing _all_ downloaded jars. > The tests would be more reflective of the behavior of a runtime environment > if the test classpath only contained jars that are bundled with the binary > release, together with explicit test dependencies. Ideally we'd use the > build/lib/ jars for the classpath since that's what gets packaged, but since > these aren't available at test compile time and should be identical to lib/ > anyway, I've used the later. > Doing so exposed a couple of references in src/java to > "org.apache.commons.lang", which is not available at runtime (should be > "org.apache.commons.lang*3*"). > Attached patch modifies the test classpath, fixes various imports in both > test/ and src/ classes, and makes some simple substitutions in the tests such > as using AbstractMap.SimpleEntry in place of > org.apache.commons.collections.keyvalue.AbstractMapEntry. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356373#comment-17356373
]
Michael Semb Wever commented on CASSANDRA-16704:
bq. My thinking is that such dependencies should then become explicit test
dependencies…
They are in places, see all those marked as "provided" or "optional"
[here|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/build.xml#L491-L657]
and
[here|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/build.xml#L749-L833].
Note, these do also include those dependencies needed solely for the sake of
compiling.
bq. …rather than allowing transitive dependencies (which may disappear or
change version when any other dependency is updated) to creep into the tests.
The {{lib/}} folder also contains transitive dependencies, just of the more
restricted scope.
> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
> Issue Type: Bug
> Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: test-with-runtime-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars.
> The tests would be more reflective of the behavior of a runtime environment
> if the test classpath only contained jars that are bundled with the binary
> release, together with explicit test dependencies. Ideally we'd use the
> build/lib/ jars for the classpath since that's what gets packaged, but since
> these aren't available at test compile time and should be identical to lib/
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to
> "org.apache.commons.lang", which is not available at runtime (should be
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both
> test/ and src/ classes, and makes some simple substitutions in the tests such
> as using AbstractMap.SimpleEntry in place of
> org.apache.commons.collections.keyvalue.AbstractMapEntry.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356350#comment-17356350
]
Angelo Polo commented on CASSANDRA-16704:
-
{quote}Tests can run against those optional dependencies that we don't bundle.
I don't think that is currently the case for any tests, but it can be.
{quote}
My thinking is that such dependencies should then become explicit test
dependencies, rather than allowing transitive dependencies (which may disappear
or change version when any other dependency is updated) to creep into the
tests. Though I don't know what 'provided' means here so hopefully I haven't
misunderstood something.
{quote}I'm not quite understanding this statement, if build/lib/jars/ doesn't
exist then neither does lib/.
{quote}
Oops, this was supposed to be build/dist/lib/.
My patch doesn't touch the IDE classpaths. Should just be a matter of removing
this
[fileset|https://github.com/apache/cassandra/blob/3282f5ecf187ecbb56b8d73ab9a9110c010898b0/build.xml#L2005].
> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
> Issue Type: Bug
> Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: test-with-runtime-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars.
> The tests would be more reflective of the behavior of a runtime environment
> if the test classpath only contained jars that are bundled with the binary
> release, together with explicit test dependencies. Ideally we'd use the
> build/lib/ jars for the classpath since that's what gets packaged, but since
> these aren't available at test compile time and should be identical to lib/
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to
> "org.apache.commons.lang", which is not available at runtime (should be
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both
> test/ and src/ classes, and makes some simple substitutions in the tests such
> as using AbstractMap.SimpleEntry in place of
> org.apache.commons.collections.keyvalue.AbstractMapEntry.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356333#comment-17356333
]
Michael Semb Wever commented on CASSANDRA-16704:
(To rehash, mostly for my own sake…)
The differences between {{build/lib/jars/}} and {{lib/}} are the following.
Only {{lib/}} contains:
- cassandra-driver-internal-only-3.25.0.zip
- futures-2.1.6-py2.py3-none-any.zip
- geomet-0.1.0.zip
- sigar-bin/
- six-1.12.0-py2.py3-none-any.zip
This is just the additional binary and python dependencies.
Only {{build/lib/jars/}} contains:
- antlr-3.5.2.jar
- assertj-core-3.15.0.jar
- byteman-4.0.6.jar
- byteman-bmunit-4.0.6.jar
- byteman-install-4.0.6.jar
- byteman-submit-4.0.6.jar
- commons-beanutils-1.7.0.jar
- commons-beanutils-core-1.8.0.jar
- commons-collections-3.2.1.jar
- commons-configuration-1.6.jar
- commons-digester-1.8.jar
- commons-el-1.0.jar
- commons-httpclient-3.0.1.jar
- commons-lang-2.4.jar
- commons-math-2.1.jar
- commons-net-1.4.1.jar
- compile-command-annotations-1.2.0.jar
- compress-lzf-0.8.4.jar
- ftplet-api-1.0.0.jar
- ftpserver-core-1.0.0.jar
- ftpserver-deprecated-1.0.0-M2.jar
- hadoop-core-1.0.3.jar
- hadoop-minicluster-1.0.3.jar
- hadoop-test-1.0.3.jar
- hsqldb-1.8.0.10.jar
- jackson-core-asl-1.0.1.jar
- jackson-mapper-asl-1.0.1.jar
- jacocoagent.jar
- jasper-compiler-5.5.12.jar
- jasper-runtime-5.5.12.jar
- jersey-core-1.0.jar
- jersey-server-1.0.jar
- jets3t-0.7.1.jar
- jetty-6.1.26.jar
- jetty-util-6.1.26.jar
- jsp-2.1-6.1.14.jar
- jsp-api-2.1-6.1.14.jar
- jsr305-2.0.2.jar
- jsr311-api-1.0.jar
- kfs-0.3.jar
- mina-core-2.0.0-M5.jar
- netty-bom-4.1.58.Final.pom
- oro-2.0.8.jar
- servlet-api-2.5-6.1.14.jar
- xmlenc-0.52.jar
This should be the difference between full build scope and just "compile" scope
(which excludes system, runtime, and provided scopes).
Maybe there are jar files here that should be removed, i.e. should not even be
part the provided scope?
bq. Tests are currently run with a classpath containing all downloaded jars.
To be accurate {{build/lib/jars/}} is intended to be the [compile+provided
scoped|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/.build/build-resolver.xml#L174]
dependency tree, while {{lib/}} is only the
[compile|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/.build/build-resolver.xml#L196]
scoped dependency tree.
bq. The tests would be more reflective of the behavior of a runtime environment
if the test classpath only contained jars that are bundled with the binary
release, together with explicit test dependencies.
Tests can run against those optional dependencies that we don't bundle. I don't
think that is currently the case for any tests, but it can be.
bq. Ideally we'd use the build/lib/ jars for the classpath since that's what
gets packaged, but since these aren't available at test compile time and should
be identical to lib/ anyway, I've used the later.
I'm not quite understanding this statement, if {{build/lib/jars/}} doesn't
exist then neither does {{lib/}}.
The {{lib/}} contents are put together later in the build cycle (under the
{{resolver-dist-lib}} target) than the {{build/lib/jars/}} contents (which are
done under the {{resolver-retrieve-build}} target). There is a valid question
here as to whether we want tests to now depend upon the {{resolver-dist-lib}}
target.
> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
> Issue Type: Bug
> Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: test-with-runtime-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars.
> The tests would be more reflective of the behavior of a runtime environment
> if the test classpath only contained jars that are bundled with the binary
> release, together with explicit test dependencies. Ideally we'd use the
> build/lib/ jars for the classpath since that's what gets packaged, but since
> these aren't available at test compile time and should be identical to lib/
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to
> "org.apache.commons.lang", which is not available at runtime (should be
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both
> test/ and src/ classes, and makes some simple substitutions in the tests such
> as using AbstractMap.SimpleEntry in place of
> org.apache.commons.collections.keyvalue.AbstractMapEntry.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355795#comment-17355795 ] Brandon Williams commented on CASSANDRA-16704: -- LGTM, [~mck] can you take a look as well? > Fix imports; run tests with packaged dependencies > - > > Key: CASSANDRA-16704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16704 > Project: Cassandra > Issue Type: Bug > Components: Build, Test/burn, Test/unit >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Fix For: 4.0.x > > Attachments: test-with-runtime-deps.patch > > > Tests are currently run with a classpath containing _all_ downloaded jars. > The tests would be more reflective of the behavior of a runtime environment > if the test classpath only contained jars that are bundled with the binary > release, together with explicit test dependencies. Ideally we'd use the > build/lib/ jars for the classpath since that's what gets packaged, but since > these aren't available at test compile time and should be identical to lib/ > anyway, I've used the later. > Doing so exposed a couple of references in src/java to > "org.apache.commons.lang", which is not available at runtime (should be > "org.apache.commons.lang*3*"). > Attached patch modifies the test classpath, fixes various imports in both > test/ and src/ classes, and makes some simple substitutions in the tests such > as using AbstractMap.SimpleEntry in place of > org.apache.commons.collections.keyvalue.AbstractMapEntry. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355174#comment-17355174 ] Brandon Williams commented on CASSANDRA-16704: -- [!https://ci-cassandra.apache.org/job/Cassandra-devbranch/828/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/828/pipeline] > Fix imports; run tests with packaged dependencies > - > > Key: CASSANDRA-16704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16704 > Project: Cassandra > Issue Type: Bug > Components: Build, Test/burn, Test/unit >Reporter: Angelo Polo >Assignee: Angelo Polo >Priority: Normal > Attachments: test-with-runtime-deps.patch > > > Tests are currently run with a classpath containing _all_ downloaded jars. > The tests would be more reflective of the behavior of a runtime environment > if the test classpath only contained jars that are bundled with the binary > release, together with explicit test dependencies. Ideally we'd use the > build/lib/ jars for the classpath since that's what gets packaged, but since > these aren't available at test compile time and should be identical to lib/ > anyway, I've used the later. > Doing so exposed a couple of references in src/java to > "org.apache.commons.lang", which is not available at runtime (should be > "org.apache.commons.lang*3*"). > Attached patch modifies the test classpath, fixes various imports in both > test/ and src/ classes, and makes some simple substitutions in the tests such > as using AbstractMap.SimpleEntry in place of > org.apache.commons.collections.keyvalue.AbstractMapEntry. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
