Bug#869777: yara tests fail on ARM32, when run on a 64bit kernel

[Steve Langasek]

Package: yara
Followup-For: Bug #869777
User: ubuntu-de...@lists.ubuntu.com
Usertags: origin-ubuntu bionic ubuntu-patch

Hi Hilko,

I've applied the following patch to yara in Ubuntu which fixes the
non-portable alignment assumptions.  This should be reasonably performant
with modern gcc on supported architectures, and on some architectures will
likely improve performance by eliminating unaligned accesses that are
supported but costly to fix up.  (This includes ARM in some configurations,
since it's possible for unaligned access to not raise SIGBUS but still be an
expensive operation.)

Please consider applying this patch in Debian.

Thanks,
-- 
Steve Langasek   Give me a lever long enough and a Free OS
Debian Developer   to set it on, and I can move the world.
Ubuntu Developerhttp://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
diff -Nru yara-3.7.1/debian/patches/no-unaligned-access.patch 
yara-3.7.1/debian/patches/no-unaligned-access.patch
--- yara-3.7.1/debian/patches/no-unaligned-access.patch 1969-12-31 
16:00:00.0 -0800
+++ yara-3.7.1/debian/patches/no-unaligned-access.patch 2018-02-04 
14:08:33.0 -0800
@@ -0,0 +1,118 @@
+Description: use alignment-safe handling
+ Casting a char* to a uint64_t* is not universally safe due to alignment
+ constraints on reads on some platforms.  Just use memcpy() instead, which
+ the compiler should optimize adequately for us.
+ .
+ Also, force alignment of our arena-allocated structures that contain 64-bit
+ elements.
+Author: Steve Langasek 
+
+Index: yara-3.7.1/libyara/exec.c
+===
+--- yara-3.7.1.orig/libyara/exec.c
 yara-3.7.1/libyara/exec.c
+@@ -227,7 +227,7 @@
+ break;
+ 
+   case OP_PUSH:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ push(r1);
+ break;
+@@ -237,13 +237,13 @@
+ break;
+ 
+   case OP_CLEAR_M:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ mem[r1.i] = 0;
+ break;
+ 
+   case OP_ADD_M:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ pop(r2);
+ if (!is_undef(r2))
+@@ -251,27 +251,27 @@
+ break;
+ 
+   case OP_INCR_M:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ mem[r1.i]++;
+ break;
+ 
+   case OP_PUSH_M:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ r1.i = mem[r1.i];
+ push(r1);
+ break;
+ 
+   case OP_POP_M:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ pop(r2);
+ mem[r1.i] = r2.i;
+ break;
+ 
+   case OP_SWAPUNDEF:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ pop(r2);
+ 
+@@ -859,7 +859,7 @@
+ break;
+ 
+   case OP_IMPORT:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ 
+ result = yr_modules_load((char*) r1.p, context);
+@@ -902,7 +902,7 @@
+ break;
+ 
+   case OP_INT_TO_DBL:
+-r1.i = *(uint64_t*)(ip);
++memcpy(, ip, sizeof(uint64_t));
+ ip += sizeof(uint64_t);
+ r2 = stack[sp - r1.i];
+ if (is_undef(r2))
+Index: yara-3.7.1/libyara/scan.c
+===
+--- yara-3.7.1.orig/libyara/scan.c
 yara-3.7.1/libyara/scan.c
+@@ -397,8 +397,11 @@
+ 
+   FAIL_ON_ERROR(yr_arena_allocate_memory(
+   context->matches_arena,
+-  sizeof(YR_MATCH),
++  sizeof(YR_MATCH) + sizeof(uint64_t) - 1,
+   (void**) _match));
++  /* force alignment */
++  new_match = (YR_MATCH *)(((size_t)new_match + sizeof(uint64_t) - 1)
++  & ~(sizeof(uint64_t) - 1));
+ 
+   new_match->data_length = yr_min(match_length, MAX_MATCH_DATA);
+ 
+@@ -500,8 +503,11 @@
+ 
+ FAIL_ON_ERROR(yr_arena_allocate_memory(
+ callback_args->context->matches_arena,
+-sizeof(YR_MATCH),
++sizeof(YR_MATCH) + sizeof(uint64_t) - 1,
+ (void**) _match));
++/* force alignment */
++new_match = (YR_MATCH *)(((size_t)new_match + sizeof(uint64_t) - 1)
++& ~(sizeof(uint64_t) - 1));
+ 
+ new_match->data_length = yr_min(match_length, MAX_MATCH_DATA);
+ 
diff -Nru yara-3.7.1/debian/patches/series yara-3.7.1/debian/patches/series
--- yara-3.7.1/debian/patches/series2018-01-16 04:45:26.0 -0800
+++ yara-3.7.1/debian/patches/series

Bug#869777: yara tests fail on ARM32, when run on a 64bit kernel

[Matthias Klose]

Package: src:yara
Version: 3.6.3+dfsg-1
Forwarded: https://github.com/VirusTotal/yara/issues/716
Tags: upstream

yara tests fail on ARM32, when run on a 64bit kernel.

This turns out to be because the codebase is full of code that assumes unaligned
access is OK, for example stuff like this
https://github.com/VirusTotal/yara/blob/master/libyara/exec.c#L224 but also
more subtly stuff like
https://github.com/VirusTotal/yara/blob/master/libyara/scan.c#L409
(new_match is only guaranteed to be 4-byte-aligned and crashes when you access
an 8 byte field).

make  check-TESTS
make[3]: Entering directory '/<>/yara-3.6.3+dfsg'
make[4]: Entering directory '/<>/yara-3.6.3+dfsg'
PASS: test-alignment
FAIL: test-rules
FAIL: test-pe
FAIL: test-elf
PASS: test-version
FAIL: test-exception
==
   yara 3.6.3: ./test-suite.log
==

# TOTAL: 6
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  4
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: test-rules


yr_rules_scan_mem: error
FAIL test-rules (exit status: 1)

FAIL: test-pe
=

yr_rules_scan_mem: error
FAIL test-pe (exit status: 1)

FAIL: test-elf
==

yr_rules_scan_mem: error
FAIL test-elf (exit status: 1)

FAIL: test-exception


Scanning for ""...
err = 4, matches = 0
Sending blocked SIGUSR1 to ourselves...
Scanning for {00 00 00 00}...
err = 4, matches = 0
Scanning for ""...
err = 4, matches = 0
Expected subprocess to be terminated by signal
Test: crash
Test: crash-no-handle
Test: blocked-signal
Test: crash-other-thread
FAIL test-exception (exit status: 1)


Testsuite summary for yara 3.6.3

# TOTAL: 6
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  4
# XPASS: 0
# ERROR: 0

See ./test-suite.log
Please report to vmalva...@virustotal.com

Makefile:1000: recipe for target 'test-suite.log' failed
make[4]: *** [test-suite.log] Error 1