[
https://issues.apache.org/jira/browse/CASSANDRA-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744177#comment-17744177
]
Jacek Lewandowski edited comment on CASSANDRA-17992 at 7/18/23 11:45 AM:
-------------------------------------------------------------------------
I've managed to fix it. The problem is not related to this particular story but
the way we produce a {{dtest jar}}. That repackages all dependencies along with
our classes in a single jar. Some of the dependencies might have digital
signatures which are extracted into {{META-INF}} as say {{*.SF}} files. We need
to remove them because they are invalid for the uber jar we produce.
(see
https://stackoverflow.com/questions/35577351/exclude-jar-from-ant-build-using-zipgroupfileset/35593838#35593838)
Therefore, there are exclusion in {{zipgroupfileset}} operations, more or less
like this:
{noformat}
<zipgroupfileset dir="${build.lib}" includes="*.jar" excludes="META-INF/*.SF"/>
{noformat}
However, this is wrong. {{exclude}} attribute denotes jar files we don't want
to repackage rather than filter on the content of the jar file. The new
BouncyCastle jars include signatures in SF and DSA files and all of them lands
in the final dtest jar regardless of that filter.
To fix it, I needed to use the following approach:
{noformat}
<zipgroupfileset dir="${build.lib}" includes="*.jar"
excludes="bc*.jar"/>
<zipgroupfileset dir="${test.lib}/jars"
includes="jimfs-1.1.jar,dtest-api-*.jar,asm-*.jar,javassist-*.jar,reflections-*.jar,semver4j-*.jar"
excludes="bc*.jar"/>
<zipgroupfileset dir="${build.dir.lib}/jars" includes="asm-*.jar"
excludes="bc*.jar"/>
<zipfileset src="${build.lib}/bcpkix-jdk15on-1.70.jar"
excludes="META-INF/*.SF,META-INF/*.DSA"/>
<zipfileset src="${build.lib}/bcprov-jdk15on-1.70.jar"
excludes="META-INF/*.SF,META-INF/*.DSA"/>
<zipfileset src="${build.lib}/bcutil-jdk15on-1.70.jar"
excludes="META-INF/*.SF,META-INF/*.DSA"/>
{noformat}
that is, I excluded bc jars from that zipgroupfileset repackaging and included
them individually. Unlike {{zipgroupfileset}}, {{zipfileset}} allows for
filtering the content of the repackaged jar file. Unfortunately, it allows to
add a single jar at a time, thus we need to provide its exact name. This
solution is not very flexible - we will have to update that task manually each
time we change certain dependencies.
I'll be looking for a better approach, but at least we know what is going on.
was (Author: jlewandowski):
I've managed to fix it. The problem is not related to this particular story but
the way we produce a {{dtest jar}}. That repackages all dependencies along with
our classes in a single jar. Some of the dependencies might have digital
signatures which are extracted into {{META-INF}} as say {{*.SF}} files. We need
to remove them because they are invalid for the uber jar we produce.
Therefore, there are exclusion in {{zipgroupfileset}} operations, more or less
like this:
{noformat}
<zipgroupfileset dir="${build.lib}" includes="*.jar" excludes="META-INF/*.SF"/>
{noformat}
However, this is wrong. {{exclude}} attribute denotes jar files we don't want
to repackage rather than filter on the content of the jar file. The new
BouncyCastle jars include signatures in SF and DSA files and all of them lands
in the final dtest jar regardless of that filter.
To fix it, I needed to use the following approach:
{noformat}
<zipgroupfileset dir="${build.lib}" includes="*.jar"
excludes="bc*.jar"/>
<zipgroupfileset dir="${test.lib}/jars"
includes="jimfs-1.1.jar,dtest-api-*.jar,asm-*.jar,javassist-*.jar,reflections-*.jar,semver4j-*.jar"
excludes="bc*.jar"/>
<zipgroupfileset dir="${build.dir.lib}/jars" includes="asm-*.jar"
excludes="bc*.jar"/>
<zipfileset src="${build.lib}/bcpkix-jdk15on-1.70.jar"
excludes="META-INF/*.SF,META-INF/*.DSA"/>
<zipfileset src="${build.lib}/bcprov-jdk15on-1.70.jar"
excludes="META-INF/*.SF,META-INF/*.DSA"/>
<zipfileset src="${build.lib}/bcutil-jdk15on-1.70.jar"
excludes="META-INF/*.SF,META-INF/*.DSA"/>
{noformat}
that is, I excluded bc jars from that zipgroupfileset repackaging and included
them individually. Unlike {{zipgroupfileset}}, {{zipfileset}} allows for
filtering the content of the repackaged jar file. Unfortunately, it allows to
add a single jar at a time, thus we need to provide its exact name. This
solution is not very flexible - we will have to update that task manually each
time we change certain dependencies.
I'll be looking for a better approach, but at least we know what is going on.
> Upgrade Netty on 5.0
> --------------------
>
> Key: CASSANDRA-17992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17992
> Project: Cassandra
> Issue Type: Task
> Components: Dependencies
> Reporter: Ekaterina Dimitrova
> Assignee: Jacek Lewandowski
> Priority: Low
> Fix For: 5.x
>
>
> I haven't been able to identify from the Netty docs which was the lowest
> version where JDK17 was added but we are about 40 versions behind in netty 4
> so I suspect we better update.
> -We need to consider there was an issue with class cast exceptions when
> building with JDK17 with newer versions of netty (the newest available in
> March 2022). For the record, we didn't see those when running CI on JDK8 and
> JDK11. We also need to carefully revise the changes between the netty
> versions. -->- CASSANDRA-18180
> Upgrading will cover also a fix in netty that was discussed in
> [this|https://the-asf.slack.com/archives/CK23JSY2K/p1665567660202989] ASF
> Slack thread.
> CC [~benedict] , [~aleksey]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
