Re: security/ghidra and libstdc++
On Fri, Aug 30, 2019 at 12:33:37AM -0400, Lawrence Teo wrote:
> On Wed, Aug 28, 2019 at 10:17:29PM -0400, Lawrence Teo wrote:
> > On Tue, Aug 27, 2019 at 10:52:52AM -, Christian Weisgerber wrote:
> > > On 2019-08-21, Stuart Henderson wrote:
> > >
> > > >> > * If you build ghidra on a system where gcc/g++/libstdc++ are
> > > >> > available, it does indeed link something against libstdc++ and
> > > >> > port-lib-depends-check indicates WANTLIB+=stdc++.
> > > >> >
> > > >> > * If you build ghidra on a system where gcc/g++/libstdc++ are NOT
> > > >> > available, it still builds fine, something is linked against
> > > >> > libc++/libc++abi/pthread and port-lib-depends-check consequently
> > > >> > indicates WANTLIB+=${COMPILER_LIBCXX}.
> > > >
> > > > Is this enough? Untested beyond "does it create the right symlink".
> > > >
> > > > --- Makefile12 Jul 2019 20:49:02 - 1.4
> > > > +++ Makefile21 Aug 2019 08:21:22 -
> > > > @@ -39,6 +39,7 @@ post-extract:
> > > > ${WRKSRC}/ghidraRun
> > > > @perl -pi -e 's,#!/bin/bash,#!${LOCALBASE}/bin/bash,' \
> > > > ${WRKSRC}/support/launch.sh
> > > > + ln -s c++ ${WRKDIR}/bin/g++
> > >
> > > This actually breaks the build:
> > >
> > > > Task :Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp FAILED
> > >
> > > FAILURE: Build failed with an exception.
> > >
> > > * What went wrong:
> > > Execution failed for task
> > > ':Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp'.
> > > > java.lang.NullPointerException (no error message)
> >
> > I have been testing and tinkering as well but haven't gotten very far.
> > Using sthen's diff and passing --stacktrace to gradle I got this output:
> [snip]
> >
> > That looks like something related to Gradle, so I did some digging in
> > the *.gradle files in the Ghidra source tree. I believe the file that
> > needs to be fixed is
> > ghidra-9.0.4/ghidra-Ghidra_9.0.4_build/GPL/nativeBuildProperties.gradle
> > but I haven't figured out exactly how yet.
>
> I think I got it. This new diff tells Gradle to build the demangler
> with Clang, and I also made it print the compiler and linker
> executables to confirm:
>
> C++ compiler is [clang++]
> Linker is [clang++]
>
> The resulting demangler_gnu is linked with libc++:
>
> nori$ ldd
> /usr/local/share/java/ghidra/GPL/DemanglerGnu/os/openbsd64/demangler_gnu
> /usr/local/share/java/ghidra/GPL/DemanglerGnu/os/openbsd64/demangler_gnu:
> StartEnd Type Open Ref GrpRef Name
> 05b4d7026000 05b4d7056000 exe 20 0
> /usr/local/share/java/ghidra/GPL/DemanglerGnu/os/openbsd64/demangler_gnu
> 05b73bfaa000 05b73c086000 rlib 01 0
> /usr/lib/libc++.so.3.0
> 05b7588a8000 05b7588ea000 rlib 02 0
> /usr/lib/libc++abi.so.1.0
> 05b6e100b000 05b6e1018000 rlib 01 0
> /usr/lib/libpthread.so.26.1
> 05b6d8f74000 05b6d8fa3000 rlib 01 0
> /usr/lib/libm.so.10.1
> 05b75bb5c000 05b75bc5 rlib 01 0
> /usr/lib/libc.so.95.1
> 05b79c817000 05b79c817000 ld.so 01 0
> /usr/libexec/ld.so
>
> I added --stacktrace to gradle's command line for easier future
> debugging.
>
> Tests/feedback welcome!
Here's a new diff that makes the port honor CXX. I also added
COMPILER="base-clang ports-clang" and confirmed that it builds with both
clangs.
ok?
Index: Makefile
===
RCS file: /cvs/ports/security/ghidra/Makefile,v
retrieving revision 1.6
diff -u -p -r1.6 Makefile
--- Makefile23 Jul 2019 02:52:58 - 1.6
+++ Makefile7 Sep 2019 02:18:06 -
@@ -7,7 +7,7 @@ COMMENT = software reverse engineering (
VERSION = 9.0.4
GHIDRA_DATE = 20190516
-REVISION = 2
+REVISION = 3
GH_ACCOUNT = NationalSecurityAgency
GH_PROJECT = ghidra
@@ -23,7 +23,7 @@ MAINTAINER = Lawrence Teo https://sourceforge.net/projects/yajsw/files/yajsw/yajsw-stable-${YAJSW_VER}/
@@ -55,6 +55,8 @@ DISTFILES += ${JAR_DISTFILES:C/$/:2/}
EXTRACT_ONLY = ${DISTNAME}.tar.gz
+COMPILER = base-clang ports-clang
+
MODULES = java
MODJAVA_VER = 11+
@@ -68,7 +70,7 @@ RUN_DEPENDS = shells/bash \
NO_TEST = Yes
-SUBST_VARS += GHIDRA_DATE VERSION WRKDIR
+SUBST_VARS += CXX GHIDRA_DATE VERSION WRKDIR
JAR_DIRS +=Features-FileFormats
JAR_DIRS +=Features-Python
@@ -94,6 +96,7 @@ post-extract:
pre-build:
cp ${FILESDIR}/repos.gradle ${WRKDIR}
${SUBST_CMD} ${WRKDIR}/repos.gradle \
+ ${WRKSRC}/GPL/nativeBuildProperties.gradle \
${WRKSRC}/Ghidra/Framework/Help/src/main/java/help/GHelpBuilder.java
mkdir ${WRKDIR}/{flatRepo,gradle,home}
.for dir in ${JAR_DIRS}
@@ -113,11 +116,13 @@
Re: security/ghidra and libstdc++
On Wed, Aug 28, 2019 at 10:17:29PM -0400, Lawrence Teo wrote:
> On Tue, Aug 27, 2019 at 10:52:52AM -, Christian Weisgerber wrote:
> > On 2019-08-21, Stuart Henderson wrote:
> >
> > >> > * If you build ghidra on a system where gcc/g++/libstdc++ are
> > >> > available, it does indeed link something against libstdc++ and
> > >> > port-lib-depends-check indicates WANTLIB+=stdc++.
> > >> >
> > >> > * If you build ghidra on a system where gcc/g++/libstdc++ are NOT
> > >> > available, it still builds fine, something is linked against
> > >> > libc++/libc++abi/pthread and port-lib-depends-check consequently
> > >> > indicates WANTLIB+=${COMPILER_LIBCXX}.
> > >
> > > Is this enough? Untested beyond "does it create the right symlink".
> > >
> > > --- Makefile 12 Jul 2019 20:49:02 - 1.4
> > > +++ Makefile 21 Aug 2019 08:21:22 -
> > > @@ -39,6 +39,7 @@ post-extract:
> > > ${WRKSRC}/ghidraRun
> > > @perl -pi -e 's,#!/bin/bash,#!${LOCALBASE}/bin/bash,' \
> > > ${WRKSRC}/support/launch.sh
> > > + ln -s c++ ${WRKDIR}/bin/g++
> >
> > This actually breaks the build:
> >
> > > Task :Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp FAILED
> >
> > FAILURE: Build failed with an exception.
> >
> > * What went wrong:
> > Execution failed for task
> > ':Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp'.
> > > java.lang.NullPointerException (no error message)
>
> I have been testing and tinkering as well but haven't gotten very far.
> Using sthen's diff and passing --stacktrace to gradle I got this output:
[snip]
>
> That looks like something related to Gradle, so I did some digging in
> the *.gradle files in the Ghidra source tree. I believe the file that
> needs to be fixed is
> ghidra-9.0.4/ghidra-Ghidra_9.0.4_build/GPL/nativeBuildProperties.gradle
> but I haven't figured out exactly how yet.
I think I got it. This new diff tells Gradle to build the demangler
with Clang, and I also made it print the compiler and linker
executables to confirm:
C++ compiler is [clang++]
Linker is [clang++]
The resulting demangler_gnu is linked with libc++:
nori$ ldd
/usr/local/share/java/ghidra/GPL/DemanglerGnu/os/openbsd64/demangler_gnu
/usr/local/share/java/ghidra/GPL/DemanglerGnu/os/openbsd64/demangler_gnu:
StartEnd Type Open Ref GrpRef Name
05b4d7026000 05b4d7056000 exe 20 0
/usr/local/share/java/ghidra/GPL/DemanglerGnu/os/openbsd64/demangler_gnu
05b73bfaa000 05b73c086000 rlib 01 0
/usr/lib/libc++.so.3.0
05b7588a8000 05b7588ea000 rlib 02 0
/usr/lib/libc++abi.so.1.0
05b6e100b000 05b6e1018000 rlib 01 0
/usr/lib/libpthread.so.26.1
05b6d8f74000 05b6d8fa3000 rlib 01 0
/usr/lib/libm.so.10.1
05b75bb5c000 05b75bc5 rlib 01 0
/usr/lib/libc.so.95.1
05b79c817000 05b79c817000 ld.so 01 0
/usr/libexec/ld.so
I added --stacktrace to gradle's command line for easier future
debugging.
Tests/feedback welcome!
Index: Makefile
===
RCS file: /cvs/ports/security/ghidra/Makefile,v
retrieving revision 1.6
diff -u -p -r1.6 Makefile
--- Makefile23 Jul 2019 02:52:58 - 1.6
+++ Makefile30 Aug 2019 03:27:26 -
@@ -7,7 +7,7 @@ COMMENT = software reverse engineering (
VERSION = 9.0.4
GHIDRA_DATE = 20190516
-REVISION = 2
+REVISION = 3
GH_ACCOUNT = NationalSecurityAgency
GH_PROJECT = ghidra
@@ -23,7 +23,7 @@ MAINTAINER = Lawrence Teo https://sourceforge.net/projects/yajsw/files/yajsw/yajsw-stable-${YAJSW_VER}/
@@ -113,11 +113,13 @@ pre-build:
cp ${DISTDIR}/yajsw-stable-${YAJSW_VER}.zip \
${WRKDIR}/ghidra.bin/Ghidra/Features/GhidraServer
cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} gradle -g ${WRKDIR}/gradle \
- --no-daemon --offline -I ${WRKDIR}/repos.gradle yajswDevUnpack
+ --no-daemon --offline --stacktrace -I ${WRKDIR}/repos.gradle \
+ yajswDevUnpack
do-build:
cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} gradle -g ${WRKDIR}/gradle \
- --no-daemon --offline -I ${WRKDIR}/repos.gradle buildGhidra
+ --no-daemon --offline --stacktrace -I ${WRKDIR}/repos.gradle \
+ buildGhidra
do-install:
${INSTALL_DATA_DIR} ${PREFIX}/share/java
Index: patches/patch-GPL_nativeBuildProperties_gradle
===
RCS file:
/cvs/ports/security/ghidra/patches/patch-GPL_nativeBuildProperties_gradle,v
retrieving revision 1.1
diff -u -p -r1.1 patch-GPL_nativeBuildProperties_gradle
--- patches/patch-GPL_nativeBuildProperties_gradle 23 Jul 2019 02:09:20
- 1.1
+++ patches/patch-GPL_nativeBuildProperties_gradle 30 Aug 2019 03:52:01
-
@@ -2,10 +2,12 @@
Re: security/ghidra and libstdc++
On Tue, Aug 27, 2019 at 10:52:52AM -, Christian Weisgerber wrote:
> On 2019-08-21, Stuart Henderson wrote:
>
> >> > * If you build ghidra on a system where gcc/g++/libstdc++ are
> >> > available, it does indeed link something against libstdc++ and
> >> > port-lib-depends-check indicates WANTLIB+=stdc++.
> >> >
> >> > * If you build ghidra on a system where gcc/g++/libstdc++ are NOT
> >> > available, it still builds fine, something is linked against
> >> > libc++/libc++abi/pthread and port-lib-depends-check consequently
> >> > indicates WANTLIB+=${COMPILER_LIBCXX}.
> >
> > Is this enough? Untested beyond "does it create the right symlink".
> >
> > --- Makefile12 Jul 2019 20:49:02 - 1.4
> > +++ Makefile21 Aug 2019 08:21:22 -
> > @@ -39,6 +39,7 @@ post-extract:
> > ${WRKSRC}/ghidraRun
> > @perl -pi -e 's,#!/bin/bash,#!${LOCALBASE}/bin/bash,' \
> > ${WRKSRC}/support/launch.sh
> > + ln -s c++ ${WRKDIR}/bin/g++
>
> This actually breaks the build:
>
> > Task :Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp FAILED
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> Execution failed for task
> ':Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp'.
> > java.lang.NullPointerException (no error message)
I have been testing and tinkering as well but haven't gotten very far.
Using sthen's diff and passing --stacktrace to gradle I got this output:
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp'.
> java.lang.NullPointerException (no error message)
* Try:
Run with --info or --debug option to get more log output. Run with --scan to
get full insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task
':Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp'.
at
org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:38)
at
org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
at
org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
at
org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
at
org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
at
org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
at
org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
at
org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
at
org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
at
org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
at
org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
at
org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
at
org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
at
org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
at
org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
at
org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Re: security/ghidra and libstdc++
On 2019-08-21, Stuart Henderson wrote:
>> > * If you build ghidra on a system where gcc/g++/libstdc++ are
>> > available, it does indeed link something against libstdc++ and
>> > port-lib-depends-check indicates WANTLIB+=stdc++.
>> >
>> > * If you build ghidra on a system where gcc/g++/libstdc++ are NOT
>> > available, it still builds fine, something is linked against
>> > libc++/libc++abi/pthread and port-lib-depends-check consequently
>> > indicates WANTLIB+=${COMPILER_LIBCXX}.
>
> Is this enough? Untested beyond "does it create the right symlink".
>
> --- Makefile 12 Jul 2019 20:49:02 - 1.4
> +++ Makefile 21 Aug 2019 08:21:22 -
> @@ -39,6 +39,7 @@ post-extract:
> ${WRKSRC}/ghidraRun
> @perl -pi -e 's,#!/bin/bash,#!${LOCALBASE}/bin/bash,' \
> ${WRKSRC}/support/launch.sh
> + ln -s c++ ${WRKDIR}/bin/g++
This actually breaks the build:
> Task :Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':Decompiler:compileDecompileOpenbsd64ExecutableDecompileCpp'.
> java.lang.NullPointerException (no error message)
--
Christian "naddy" Weisgerber na...@mips.inka.de
Re: security/ghidra and libstdc++
On 2019/08/20 22:43, Lawrence Teo wrote:
> On Tue, Aug 20, 2019 at 04:32:48PM +0200, Christian Weisgerber wrote:
> > A few days ago, I did a test bulk build on amd64 with gcc, g++, and
> > libstdc++ removed from the system. Ports should not use these any
> > longer on clang platforms.
> >
> > security/ghidra failed to build right away since it has stdc++ in
> > WANTLIB. Further testing shows:
> >
> > * If you build ghidra on a system where gcc/g++/libstdc++ are
> > available, it does indeed link something against libstdc++ and
> > port-lib-depends-check indicates WANTLIB+=stdc++.
> >
> > * If you build ghidra on a system where gcc/g++/libstdc++ are NOT
> > available, it still builds fine, something is linked against
> > libc++/libc++abi/pthread and port-lib-depends-check consequently
> > indicates WANTLIB+=${COMPILER_LIBCXX}.
> >
> > There must be some magic in the build where it first tries to use
> > g++ and falls back to c++. Unfortunately, the actual build commands
> > are hidden and the build is run by some Java tool, so it's not
> > obvious what's going on.
> >
> > The port should honor CXX or at least not use g++. Maybe Lawrence
> > or somebody else can take a look...
> >
> > --
> > Christian "naddy" Weisgerber na...@mips.inka.de
>
> I took a look, and you're right that it's not obvious at all.
>
> I'm not familiar with that part of the build process so I'll have a lot
> of poking around to do. :) My current suspect is a hardcoded "g++"
> inside jython-standalone-2.7.1.jar.
>
> If someone else wants to look around as well, help is very welcome!
>
Is this enough? Untested beyond "does it create the right symlink".
Index: Makefile
===
RCS file: /cvs/ports/security/ghidra/Makefile,v
retrieving revision 1.4
diff -u -p -r1.4 Makefile
--- Makefile12 Jul 2019 20:49:02 - 1.4
+++ Makefile21 Aug 2019 08:21:22 -
@@ -39,6 +39,7 @@ post-extract:
${WRKSRC}/ghidraRun
@perl -pi -e 's,#!/bin/bash,#!${LOCALBASE}/bin/bash,' \
${WRKSRC}/support/launch.sh
+ ln -s c++ ${WRKDIR}/bin/g++
do-install:
${INSTALL_DATA_DIR} ${PREFIX}/share/java/ghidra
Re: security/ghidra and libstdc++
On Tue, Aug 20, 2019 at 04:32:48PM +0200, Christian Weisgerber wrote:
> A few days ago, I did a test bulk build on amd64 with gcc, g++, and
> libstdc++ removed from the system. Ports should not use these any
> longer on clang platforms.
>
> security/ghidra failed to build right away since it has stdc++ in
> WANTLIB. Further testing shows:
>
> * If you build ghidra on a system where gcc/g++/libstdc++ are
> available, it does indeed link something against libstdc++ and
> port-lib-depends-check indicates WANTLIB+=stdc++.
>
> * If you build ghidra on a system where gcc/g++/libstdc++ are NOT
> available, it still builds fine, something is linked against
> libc++/libc++abi/pthread and port-lib-depends-check consequently
> indicates WANTLIB+=${COMPILER_LIBCXX}.
>
> There must be some magic in the build where it first tries to use
> g++ and falls back to c++. Unfortunately, the actual build commands
> are hidden and the build is run by some Java tool, so it's not
> obvious what's going on.
>
> The port should honor CXX or at least not use g++. Maybe Lawrence
> or somebody else can take a look...
>
> --
> Christian "naddy" Weisgerber na...@mips.inka.de
I took a look, and you're right that it's not obvious at all.
I'm not familiar with that part of the build process so I'll have a lot
of poking around to do. :) My current suspect is a hardcoded "g++"
inside jython-standalone-2.7.1.jar.
If someone else wants to look around as well, help is very welcome!
security/ghidra and libstdc++
A few days ago, I did a test bulk build on amd64 with gcc, g++, and
libstdc++ removed from the system. Ports should not use these any
longer on clang platforms.
security/ghidra failed to build right away since it has stdc++ in
WANTLIB. Further testing shows:
* If you build ghidra on a system where gcc/g++/libstdc++ are
available, it does indeed link something against libstdc++ and
port-lib-depends-check indicates WANTLIB+=stdc++.
* If you build ghidra on a system where gcc/g++/libstdc++ are NOT
available, it still builds fine, something is linked against
libc++/libc++abi/pthread and port-lib-depends-check consequently
indicates WANTLIB+=${COMPILER_LIBCXX}.
There must be some magic in the build where it first tries to use
g++ and falls back to c++. Unfortunately, the actual build commands
are hidden and the build is run by some Java tool, so it's not
obvious what's going on.
The port should honor CXX or at least not use g++. Maybe Lawrence
or somebody else can take a look...
--
Christian "naddy" Weisgerber na...@mips.inka.de
