Re: [Qemu-devel] [PATCH] Fix TLS support on x86
Fabrice Bellard schrieb: On Tue, 2007-11-13 at 19:44 +0100, Stefan Weil wrote: Hi, I'd like to test user mode emulation on MIPS host. Do you have newer versions of your patches which match current CVS HEAD? TLS support is still missing there, so QEMU user mode emulation is not really usable without working patches. I don't know about anyone else, but I have some patches for TLS for arm and i386. The patches are significantly out of date with the recent linux-user changes. I won't be updating them until I finish with the current set of changes to linux-user. I'll try to merge and improve the i386 TLS patch ASAP. Moreover, after having looked again at the linux-user code and at the regression tests, my conclusion is that the linux-user QEMU targets must be statically linked and that it should not depend on host libraries such as the C library. Fabrice. Thank you, Fabrice. With your latest changes in QEMU CVS, the user mode emulation qemu-i386 works again: * i386 host: statically linked hello, Debian echo, Debian top, Debian ls ok * mips host (le and be QEMU Malta emulation): statically linked hello, Debian echo, Debian top ok Debian ls fails (segmentation fault) Debian top complains about 2.4+ kernel w/o ELF notes - this might be a missing feature in the linux user mode emulation. Mips hosts need patches (published but still missing in CVS HEAD) for user mode emulation. Regards Stefan
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
Hi, I'd like to test user mode emulation on MIPS host. Do you have newer versions of your patches which match current CVS HEAD? TLS support is still missing there, so QEMU user mode emulation is not really usable without working patches. Regards Stefan Alexander Graf schrieb: Hi, these are the updated patches for TLS support: qemu-cvs-futex.patch This patches futex support into qemu-user. It is basically done by David Woodhouse and I implemented FUTEX_WAKE_OP because actually one application did not work without (I don't really remember which one though). If FUTEX_WAKE_OP gets triggered qemu throws a warning though so if anyone experiences problems with it and it works without we should disable it then. qemu-cvs-sched_getaffinity.patch Flash9 needs sys_get_getaffinity to work properly. As far as I can tell there should be no need for endianness-conversion, because the information is written bit-wise. qemu-cvs-tls.patch implements set_thread_area for x86 and modifies the do_clone function, so TLS is evaluated. This is 90% done by David Woodhouse, I only changed it so it works for me (TID setters, proper segment register setters, fork() fix, made clone() work). Alex
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
On Tue, 2007-11-13 at 19:44 +0100, Stefan Weil wrote: Hi, I'd like to test user mode emulation on MIPS host. Do you have newer versions of your patches which match current CVS HEAD? TLS support is still missing there, so QEMU user mode emulation is not really usable without working patches. I don't know about anyone else, but I have some patches for TLS for arm and i386. The patches are significantly out of date with the recent linux-user changes. I won't be updating them until I finish with the current set of changes to linux-user. Alexander Graf schrieb: Hi, these are the updated patches for TLS support: qemu-cvs-futex.patch This patches futex support into qemu-user. It is basically done by David Woodhouse and I implemented FUTEX_WAKE_OP because actually one application did not work without (I don't really remember which one though). If FUTEX_WAKE_OP gets triggered qemu throws a warning though so if anyone experiences problems with it and it works without we should disable it then. qemu-cvs-sched_getaffinity.patch Flash9 needs sys_get_getaffinity to work properly. As far as I can tell there should be no need for endianness-conversion, because the information is written bit-wise. qemu-cvs-tls.patch implements set_thread_area for x86 and modifies the do_clone function, so TLS is evaluated. This is 90% done by David Woodhouse, I only changed it so it works for me (TID setters, proper segment register setters, fork() fix, made clone() work). Alex
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
Thayne Harbaugh wrote: On Tue, 2007-11-13 at 19:44 +0100, Stefan Weil wrote: Hi, I'd like to test user mode emulation on MIPS host. Do you have newer versions of your patches which match current CVS HEAD? TLS support is still missing there, so QEMU user mode emulation is not really usable without working patches. I don't know about anyone else, but I have some patches for TLS for arm and i386. The patches are significantly out of date with the recent linux-user changes. I won't be updating them until I finish with the current set of changes to linux-user. I'll try to merge and improve the i386 TLS patch ASAP. Moreover, after having looked again at the linux-user code and at the regression tests, my conclusion is that the linux-user QEMU targets must be statically linked and that it should not depend on host libraries such as the C library. Fabrice.
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
Hi,
these are the updated patches for TLS support:
qemu-cvs-futex.patch
This patches futex support into qemu-user. It is basically done by David
Woodhouse and I implemented FUTEX_WAKE_OP because actually one
application did not work without (I don't really remember which one
though). If FUTEX_WAKE_OP gets triggered qemu throws a warning though so
if anyone experiences problems with it and it works without we should
disable it then.
qemu-cvs-sched_getaffinity.patch
Flash9 needs sys_get_getaffinity to work properly. As far as I can tell
there should be no need for endianness-conversion, because the
information is written bit-wise.
qemu-cvs-tls.patch
implements set_thread_area for x86 and modifies the do_clone function,
so TLS is evaluated. This is 90% done by David Woodhouse, I only changed
it so it works for me (TID setters, proper segment register setters,
fork() fix, made clone() work).
Alex
Index: qemu/linux-user/syscall.c
===
--- qemu.orig/linux-user/syscall.c
+++ qemu/linux-user/syscall.c
@@ -17,6 +17,8 @@
* along with this program; if not, write to the Free Software
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+
+#define __user
#include stdlib.h
#include stdio.h
#include stdarg.h
@@ -60,6 +62,7 @@
#define tchars host_tchars /* same as target */
#define ltchars host_ltchars /* same as target */
+#include linux/futex.h
#include linux/termios.h
#include linux/unistd.h
#include linux/utsname.h
@@ -2554,6 +2557,91 @@ static inline void host_to_target_timesp
unlock_user_struct(target_ts, target_addr, 1);
}
+#ifdef BSWAP_NEEDED
+static int futex_op(int oldval, int op, int oparg)
+{
+ int retval = oparg;
+ switch(op) {
+ case FUTEX_OP_SET: break;
+ case FUTEX_OP_ADD: retval += oparg; break;
+ case FUTEX_OP_OR: retval |= oparg; break;
+ case FUTEX_OP_ANDN: retval = oparg; break;
+ case FUTEX_OP_XOR: retval ^= oparg; break;
+ }
+ return retval;
+}
+
+static int futex_cmp(int oldval, int cmp, int cmparg)
+{
+ switch(cmp) {
+ case FUTEX_OP_CMP_EQ: return oldval == cmparg;
+ case FUTEX_OP_CMP_NE: return oldval != cmparg;
+ case FUTEX_OP_CMP_LT: return oldval cmparg;
+ case FUTEX_OP_CMP_LE: return oldval = cmparg;
+ case FUTEX_OP_CMP_GT: return oldval cmparg;
+ case FUTEX_OP_CMP_GE: return oldval = cmparg;
+ }
+ return -1;
+}
+#endif
+
+static long do_futex(target_ulong uaddr, int op, uint32_t val,
+target_ulong utime, target_ulong uaddr2,
+uint32_t val3)
+{
+ struct timespec host_utime;
+ unsigned long val2 = utime;
+ long retval;
+
+ if (utime (op == FUTEX_WAIT || op == FUTEX_LOCK_PI)) {
+ target_to_host_timespec(host_utime, utime);
+ val2 = (unsigned long)host_utime;
+ }
+
+#ifdef BSWAP_NEEDED
+ switch(op) {
+ case FUTEX_CMP_REQUEUE:
+ val3 = tswap32(val3);
+ case FUTEX_REQUEUE:
+ val2 = tswap32(val2);
+ case FUTEX_WAIT:
+ case FUTEX_WAKE:
+ case FUTEX_WAKE_OP:
+ val = tswap32(val);
+ case FUTEX_LOCK_PI: /* This one's icky, but comes out OK */
+ case FUTEX_UNLOCK_PI:
+ break;
+ default:
+ gemu_log(qemu: Unsupported futex op %d\n, op);
+ return -ENOSYS;
+ }
+ if (op == FUTEX_WAKE_OP) {
+ /* Need to munge the secondary operation (val3) */
+ gemu_log(qemu: Tricky FUTEX_WAKE_OP - trying to emulate it\n);
+ val3 = tswap32(val3);
+ int op2 = (val3 28) 0xf;
+ int cmp = (val3 24) 0xf;
+ int oparg = (val3 12) 0xfff;
+ int cmparg = val3 0xfff;
+ int shift = val3 (FUTEX_OP_OPARG_SHIFT 28);
+ int oldval = tget32(uaddr2);
+ if (shift)
+ oparg = 1 oparg;
+
+ tput32(uaddr2,futex_op(oldval, op2, oparg));
+ retval = syscall(__NR_futex, g2h(uaddr), FUTEX_WAKE, val, 0, 0, 0);
+ if(futex_cmp(oldval, cmp, cmparg)) {
+ retval = syscall(__NR_futex, g2h(uaddr2), FUTEX_WAKE, val2, 0, 0, 0);
+ }
+ } else {
+ retval = syscall(__NR_futex, g2h(uaddr), op, val, val2, g2h(uaddr2), val3);
+ }
+#else
+ retval = syscall(__NR_futex, g2h(uaddr), op, val, val2, g2h(uaddr2), val3);
+#endif
+ return retval;
+}
+
long do_syscall(void *cpu_env, int num, long arg1, long arg2, long arg3,
long arg4, long arg5, long arg6)
{
@@ -4713,6 +4801,11 @@ long do_syscall(void *cpu_env, int num,
}
#endif
+#ifdef TARGET_NR_futex
+case TARGET_NR_futex:
+ret = get_errno(do_futex(arg1, arg2, arg3, arg4, arg5, arg6));
+break;
+#endif
#if defined(TARGET_NR_set_tid_address) defined(__NR_set_tid_address)
case TARGET_NR_set_tid_address:
ret
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
On Jun 21, 2007, at 7:31 AM, David Woodhouse wrote: On Wed, 2007-06-20 at 18:42 +0200, Alexander Graf wrote: implements futexes (this is mostly done by David Woodhouse as well, FUTEX_WAKE_OP done by me) #ifdef BSWAP_NEEDED, only FUTEX_OP_CMP_EQ and FUTEX_OP_CMP_NE will work as expected. If we want to do the rest then we'll need to implement FUTEX_OP_CMP_LT_WRONGENDIAN c in the kernel. Tell me if I'm wrong there, but as far as I could tell from the documentation by Ulrich Drapper (http://people.redhat.com/drepper/ futex.pdf) and the kernel source, FUTEX_WAKE_OP can be easily expressed as two conditional FUTEX_WAKE calls. Because the only real endianness problems that exist are in the conditional checking, we should be on the safe side if that is done in qemu (which is what I was implementing). Alex
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
On Thu, Jun 21, 2007 at 01:31:07PM +0800, David Woodhouse wrote: On Wed, 2007-06-20 at 18:42 +0200, Alexander Graf wrote: implements futexes (this is mostly done by David Woodhouse as well, FUTEX_WAKE_OP done by me) #ifdef BSWAP_NEEDED, only FUTEX_OP_CMP_EQ and FUTEX_OP_CMP_NE will work as expected. If we want to do the rest then we'll need to implement FUTEX_OP_CMP_LT_WRONGENDIAN c in the kernel. Or maybe, since we don't do set_robust_list (and would need wrong-endian support in the kernel for that too), we can assume that it's all in-process, and hence all _within_ qemu, and we could actually implement the futex stuff entirely within qemu with qemu's own locking? For now I think the safer option is just to leave FUTEX_WAKE_OP unimplemented. Jakub, what do you think? FUTEX_WAKE_OP is just an optimization and the only op glibc uses is FUTEX_OP_CLEAR_WAKE_IF_GT_ONE, so something that would need kernel help. But all glibcs so far if syscall (SYS_futex, ... FUTEX_WAKE_OP, ...) fails just fall back to syscall (SYS_futex, ... FUTEX_WAKE, ...) + unlock. So I think it is safe to leave FUTEX_WAKE_OP not supported for BSWAP_NEEDED. Jakub
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
Thiemo Seufer wrote:
Alexander Graf wrote:
Hi,
this patch is based on the NPTL/TLS patch, David Woodhouse sent to the
list some months ago, which unfortulately did not work for me, so these
are the fixes needed to get it working. After all there is a certain
chance, that I got something wrong but basically it does the following:
1. Implement the tg_kill syscall
2. Set the GS shadow register according to the information
set_thread_area receives. I'm not sure about that part, but using the
qemu internal functions (cpu_x86_load_seg) did not work for me.
3. Implement the new (2.5.xx) TID setting features of clone()
4. Use clone() for forking, since fork() did not always work for me
(especially when using TLS)
Please split the patch per-feature, and make sure it applies to CVS head.
(It fails to apply because it appears to depend on another patch which
adds fadvise64 support.) Also, keep the code formatting style the same as
the surrounding code (line length, space vs. tabs, whitespace around
parentheses, ... ).
Thiemo
These are all of my patches so far, split as far as I thought might be
helpful and including the patch by David Woodhouse. Basically they are
independent of each other, because they affect similar areas of the
sources it's best to apply them in order though. I believe, I sent the
fadvise64-patch to the list as well some time ago. It is invalid and
everything should work without it as well though.
Would it be better to tgz the patches if I plan to send them to the list?
I hope there are no formatting mistakes left and everything applies
properly to CVS.
qemu-cvs-tgkill.patch
implements the tgkill syscall.
qemu-cvs-tls.patch
implements set_thread_area for x86 and modifies the do_clone function,
so TLS is evaluated. This is 90% done by David Woodhouse, I only changed
it so it works for me (TID setters, proper segment register setters,
fork() fix).
qemu-cvs-futex.patch
implements futexes (this is mostly done by David Woodhouse as well,
FUTEX_WAKE_OP done by me)
qemu-cvs-set_robust_list.patch
set_robust_list is only implemented in 2.6.16, so it does not hurt to
ignore that one
qemu-cvs-sched_getaffinity.patch
Flash9 needs sys_get_getaffinity to work properly. As far as I can tell
there should be no need for endianness-conversion, because the
information is written bit-wise.
qemu-cvs-wine.patch
Wine checks if %FS is 0, and if it is initializes it properly. This
usually is the case with a normal i586 system (26 in qemu-i386), so we
should emulate that behavior as well. I'm not sure if this is the way to
go though, somehow the code looks ugly ;-)
Index: qemu/linux-user/syscall.c
===
--- qemu.orig/linux-user/syscall.c
+++ qemu/linux-user/syscall.c
@@ -17,6 +17,8 @@
* along with this program; if not, write to the Free Software
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+
+#define __user
#include stdlib.h
#include stdio.h
#include stdarg.h
@@ -60,6 +62,7 @@
#define tchars host_tchars /* same as target */
#define ltchars host_ltchars /* same as target */
+#include linux/futex.h
#include linux/termios.h
#include linux/unistd.h
#include linux/utsname.h
@@ -2556,6 +2559,91 @@ static inline void host_to_target_timesp
unlock_user_struct(target_ts, target_addr, 1);
}
+#ifdef BSWAP_NEEDED
+static int futex_op(int oldval, int op, int oparg)
+{
+ int retval = oparg;
+ switch(op) {
+ case FUTEX_OP_SET: break;
+ case FUTEX_OP_ADD: retval += oparg; break;
+ case FUTEX_OP_OR: retval |= oparg; break;
+ case FUTEX_OP_ANDN: retval = oparg; break;
+ case FUTEX_OP_XOR: retval ^= oparg; break;
+ }
+ return retval;
+}
+
+static int futex_cmp(int oldval, int cmp, int cmparg)
+{
+ switch(cmp) {
+ case FUTEX_OP_CMP_EQ: return oldval == cmparg;
+ case FUTEX_OP_CMP_NE: return oldval != cmparg;
+ case FUTEX_OP_CMP_LT: return oldval cmparg;
+ case FUTEX_OP_CMP_LE: return oldval = cmparg;
+ case FUTEX_OP_CMP_GT: return oldval cmparg;
+ case FUTEX_OP_CMP_GE: return oldval = cmparg;
+ }
+ return -1;
+}
+#endif
+
+static long do_futex(target_ulong uaddr, int op, uint32_t val,
+target_ulong utime, target_ulong uaddr2,
+uint32_t val3)
+{
+ struct timespec host_utime;
+ unsigned long val2 = utime;
+ long retval;
+
+ if (utime (op == FUTEX_WAIT || op == FUTEX_LOCK_PI)) {
+ target_to_host_timespec(host_utime, utime);
+ val2 = (unsigned long)host_utime;
+ }
+
+#ifdef BSWAP_NEEDED
+ switch(op) {
+ case FUTEX_CMP_REQUEUE:
+ val3 = tswap32(val3);
+ case FUTEX_REQUEUE:
+ val2 = tswap32(val2);
+ case FUTEX_WAIT:
+ case FUTEX_WAKE:
+ case FUTEX_WAKE_OP:
+ val = tswap32(val);
+ case FUTEX_LOCK_PI: /* This one's icky, but comes out OK */
+ case
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
On Wed, 2007-06-20 at 18:42 +0200, Alexander Graf wrote: implements futexes (this is mostly done by David Woodhouse as well, FUTEX_WAKE_OP done by me) #ifdef BSWAP_NEEDED, only FUTEX_OP_CMP_EQ and FUTEX_OP_CMP_NE will work as expected. If we want to do the rest then we'll need to implement FUTEX_OP_CMP_LT_WRONGENDIAN c in the kernel. Or maybe, since we don't do set_robust_list (and would need wrong-endian support in the kernel for that too), we can assume that it's all in-process, and hence all _within_ qemu, and we could actually implement the futex stuff entirely within qemu with qemu's own locking? For now I think the safer option is just to leave FUTEX_WAKE_OP unimplemented. Jakub, what do you think? -- dwmw2
Re: [Qemu-devel] [PATCH] Fix TLS support on x86
Alexander Graf wrote: Hi, this patch is based on the NPTL/TLS patch, David Woodhouse sent to the list some months ago, which unfortulately did not work for me, so these are the fixes needed to get it working. After all there is a certain chance, that I got something wrong but basically it does the following: 1. Implement the tg_kill syscall 2. Set the GS shadow register according to the information set_thread_area receives. I'm not sure about that part, but using the qemu internal functions (cpu_x86_load_seg) did not work for me. 3. Implement the new (2.5.xx) TID setting features of clone() 4. Use clone() for forking, since fork() did not always work for me (especially when using TLS) Please split the patch per-feature, and make sure it applies to CVS head. (It fails to apply because it appears to depend on another patch which adds fadvise64 support.) Also, keep the code formatting style the same as the surrounding code (line length, space vs. tabs, whitespace around parentheses, ... ). Thiemo
[Qemu-devel] [PATCH] Fix TLS support on x86
Hi,
this patch is based on the NPTL/TLS patch, David Woodhouse sent to the
list some months ago, which unfortulately did not work for me, so these
are the fixes needed to get it working. After all there is a certain
chance, that I got something wrong but basically it does the following:
1. Implement the tg_kill syscall
2. Set the GS shadow register according to the information
set_thread_area receives. I'm not sure about that part, but using the
qemu internal functions (cpu_x86_load_seg) did not work for me.
3. Implement the new (2.5.xx) TID setting features of clone()
4. Use clone() for forking, since fork() did not always work for me
(especially when using TLS)
I'm open for suggestions. Using this patch all single-threaded and most
simple multi-threaded current applications (glibc 2.5) do work.
Everything bigger however does not. If anyone on this list is more into
these things, I would very much appreciate any help.
Regards,
Alexander Graf
Index: qemu-0.9.0/linux-user/i386/syscall_nr.h
===
--- qemu-0.9.0.orig/linux-user/i386/syscall_nr.h
+++ qemu-0.9.0/linux-user/i386/syscall_nr.h
@@ -271,5 +271,6 @@
#define TARGET_NR_clock_getres (TARGET_NR_timer_create+7)
#define TARGET_NR_clock_nanosleep (TARGET_NR_timer_create+8)
+#define TARGET_NR_tgkill 270
#define TARGET_NR_utimes 271
#define TARGET_NR_fadvise64_64 272
Index: qemu-0.9.0/linux-user/syscall.c
===
--- qemu-0.9.0.orig/linux-user/syscall.c
+++ qemu-0.9.0/linux-user/syscall.c
@@ -145,6 +145,8 @@ type name (type1 arg1,type2 arg2,type3 a
#define __NR_sys_rt_sigqueueinfo __NR_rt_sigqueueinfo
#define __NR_sys_syslog __NR_syslog
#define __NR_sys_fadvise64 __NR_fadvise64
+#define __NR_sys_tgkill __NR_tgkill
+#define __NR_sys_clone __NR_clone
#if defined(__alpha__) || defined (__ia64__) || defined(__x86_64__)
#define __NR__llseek __NR_lseek
@@ -166,6 +168,8 @@ _syscall5(int, _llseek, uint, fd, ulon
_syscall3(int,sys_rt_sigqueueinfo,int,pid,int,sig,siginfo_t *,uinfo)
_syscall3(int,sys_syslog,int,type,char*,bufp,int,len)
_syscall4(int,sys_fadvise64,int,fd,loff_t,offset,loff_t,len,int,advice)
+_syscall3(int,sys_tgkill,int,tgid,int,pid,int,sig)
+_syscall5(int,sys_clone, int, flags, void *, child_stack, int *, parent_tidptr, struct user_desc *, newtls, int *, child_tidptr)
#ifdef __NR_exit_group
_syscall1(int,exit_group,int,error_code)
#endif
@@ -1699,7 +1704,7 @@ int do_modify_ldt(CPUX86State *env, int
return ret;
}
-int do_set_thread_area(CPUX86State *env, target_ulong ptr)
+int do_set_thread_area(CPUX86State *env, target_ulong ptr, int set_shadow_registers)
{
uint64_t *gdt_table = g2h(env-gdt.base);
struct target_modify_ldt_ldt_s ldt_info;
@@ -1708,6 +1713,7 @@ int do_set_thread_area(CPUX86State *env,
int seg_not_present, useable;
uint32_t *lp, entry_1, entry_2;
int i;
+SegmentCache *sc = env-segs[R_GS];
lock_user_struct(target_ldt_info, ptr, 1);
ldt_info.entry_number = tswap32(target_ldt_info-entry_number);
@@ -1767,6 +1767,12 @@ int do_set_thread_area(CPUX86State *env,
(useable 20) |
0x7000;
+if(set_shadow_registers) {
+sc-selector = env-regs[R_GS];
+sc-base = ldt_info.base_addr;
+sc-limit = ldt_info.limit;
+sc-flags = entry_2;
+}
/* Install the new entry ... */
install:
lp = (uint32_t *)(gdt_table + ldt_info.entry_number);
@@ -1779,20 +1802,21 @@ install:
thread/process */
#define NEW_STACK_SIZE 8192
-static int clone_func(void *arg)
+static int clone_func(CPUState *cloneenv)
{
-CPUState *env = arg;
-cpu_loop(env);
+cpu_loop(cloneenv);
/* never exits */
return 0;
}
-int do_fork(CPUState *env, unsigned int flags, unsigned long newsp)
+int do_fork(CPUState *env, unsigned int flags, target_ulong newsp, target_ulong parent_tidptr, target_ulong newtls, target_ulong child_tidptr)
{
int ret;
+int cpu_index;
+unsigned long parent_tid=gettid();
TaskState *ts;
uint8_t *new_stack;
-CPUState *new_env;
+CPUState *new_env, *next_cpu;
#if defined(TARGET_I386)
uint64_t *new_gdt_table;
#endif
@@ -1807,9 +1835,14 @@ int do_fork(CPUState *env, unsigned int
/* add in task state list */
ts-next = first_task_state;
first_task_state = ts;
-/* we create a new CPU instance. */
-new_env = cpu_init();
-memcpy(new_env, env, sizeof(CPUState));
+/* we create a new CPU instance. (cpu_copy() in cvs) */
+ new_env = cpu_init();
+ /* preserve chaining and index */
+ next_cpu = new_env-next_cpu;
+ cpu_index = new_env-cpu_index;
+ memcpy(new_env, env,
