Hello,

We have noticed random deadlocks in malloc/free when using uClibc-ng +
Thumb1 + NPTL compiled for ARMv6 in multi-threaded environment.


Steps to reproduce

1. Build Buildroot 2019.02.9 SDK with the following configuration:

     BR2_arm=y
     BR2_arm1176jz_s=y
     BR2_ARM_INSTRUCTIONS_THUMB=y
     BR2_SHARED_STATIC_LIBS=y
     BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
     BR2_TOOLCHAIN_BUILDROOT_USE_SSP=y
     BR2_TOOLCHAIN_BUILDROOT_CXX=y

2. Cross compile the sample (see below) application with the SDK using
    static linking:

     $ arm-buildroot-linux-uclibcgnueabi-gcc -static -g deadlock.c \
     -lpthread -o deadlock

3. Run the application with at least two threads

3.1. Run the application using QEMU static emulation:

     $ qemu-arm-static ./deadlock 4 1000000

3.2. Run the application on a device with ARMv6 support (ex. any RPi
      with Raspbian image):

     $ ./deadlock 4 1000000

Expected results:

The application should always exit cleanly.

Actual results:

The application often deadlocks in both cases.


Notes:

1. The bug is also reproducible with latest uClibc-ng (v1.0.32)

2. According to our tests the endianness does not matter

3. The callstack when deadlock appears:

(gdb) info threads
   Id   Target Id         Frame
* 1    LWP 11653 "mutex" 0x000165b2 in pthread_join ()
   2    LWP 11656 "mutex" 0x00014cac in __lll_lock_wait ()
   3    LWP 11657 "mutex" 0x00014ce2 in __lll_lock_wait ()
   4    LWP 11658 "mutex" 0x00014cac in __lll_lock_wait ()
   5    LWP 11659 "mutex" 0x00014cac in __lll_lock_wait ()

(gdb) t 1
[Switching to thread 1 (LWP 11653)]
#0  0x000165b2 in pthread_join ()

(gdb) bt
#0  0x000165b2 in pthread_join ()
#1  0x00010396 in main (argc=3, argv=0x7efff784) at deadlock.c:42

(gdb) t 2
[Switching to thread 2 (LWP 11656)]
#0  0x00014cac in __lll_lock_wait ()

(gdb) bt
#0  0x00014cac in __lll_lock_wait ()
#1  0x00016b7e in pthread_mutex_lock ()
#2  0x0001199e in __libc_malloc ()
#3  0x00010278 in do_request (data=0x1) at deadlock.c:13
#4  0x00015b36 in start_thread ()
#5  0x000173f4 in clone ()

(gdb) t 3
[Switching to thread 3 (LWP 11657)]
#0  0x00014ce2 in __lll_lock_wait ()

(gdb) bt
#0  0x00014ce2 in __lll_lock_wait ()
#1  0x00016b7e in pthread_mutex_lock ()
#2  0x000117c2 in __libc_free ()
#3  0x00010284 in do_request (data=0x2) at deadlock.c:15
#4  0x00015b36 in start_thread ()
#5  0x000173f4 in clone ()

(gdb) t 4
[Switching to thread 4 (LWP 11658)]
#0  0x00014cac in __lll_lock_wait ()

(gdb) bt
#0  0x00014cac in __lll_lock_wait ()
#1  0x00016b7e in pthread_mutex_lock ()
#2  0x0001199e in __libc_malloc ()
#3  0x00010278 in do_request (data=0x3) at deadlock.c:13
#4  0x00015b36 in start_thread ()
#5  0x000173f4 in clone ()

(gdb) t 5
[Switching to thread 5 (LWP 11659)]
#0  0x00014cac in __lll_lock_wait ()

(gdb) bt
#0  0x00014cac in __lll_lock_wait ()
#1  0x00016b7e in pthread_mutex_lock ()
#2  0x0001199e in __libc_malloc ()
#3  0x00010278 in do_request (data=0x4) at deadlock.c:13
#4  0x00015b36 in start_thread ()
#5  0x000173f4 in clone ()

Sample application:

#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>

int count;

void *do_request(void *data)
{
     for (int i = 0; i < count; i++) {
         if (i % 1000 == 0)
             printf("Alloc for %p count %d\n", data, i);
         char *buffer = malloc(32 * sizeof(char));

         free(buffer);
     }
     return NULL;
}

int main(int argc, char *argv[])
{
     int i, rc;
     int threads_count = atoi(argv[1]);

     count = atoi(argv[2]);

     pthread_t threads[threads_count];
     printf("Creating %d threads\n", threads_count);

     for (i = 0; i < threads_count; i++) {
         rc = pthread_create(&threads[i], NULL, do_request, (int*)(i+1));

         if (rc) {
             printf("Error creating thread\n");
             exit(1);
         }
     }

     for (i = 0; i < threads_count; i++)
         if (pthread_join(threads[i], NULL)) {
             printf("Error joining thread\n");
             exit(1);
         }

     printf("Done. \n");
     return 0;
}


Best regards,
Gyula Farkas
_______________________________________________
devel mailing list
[email protected]
https://mailman.uclibc-ng.org/cgi-bin/mailman/listinfo/devel

Reply via email to