changeset 5ea85692a53e in /z/repo/gem5
details: http://repo.gem5.org/gem5?cmd=changeset;node=5ea85692a53e
description:
syscall_emul: [patch 13/22] add system call retry capability
This changeset adds functionality that allows system calls to retry
without
affecting thread context state such as the program counter or register
values
for the associated thread context (when system calls return with a retry
fault).
This functionality is needed to solve problems with blocking system
calls
in multi-process or multi-threaded simulations where information is
passed
between processes/threads. Blocking system calls can cause deadlock
because
the simulator itself is single threaded. There is only a single thread
servicing the event queue which can cause deadlock if the thread hits a
blocking system call instruction.
To illustrate the problem, consider two processes using the
producer/consumer
sharing model. The processes can use file descriptors and the read and
write
calls to pass information to one another. If the consumer calls the
blocking
read system call before the producer has produced anything, the call
will
block the event queue (while executing the system call instruction) and
deadlock the simulation.
The solution implemented in this changeset is to recognize that the
system
calls will block and then generate a special retry fault. The fault will
be sent back up through the function call chain until it is exposed to
the
cpu model's pipeline where the fault becomes visible. The fault will
trigger
the cpu model to replay the instruction at a future tick where the call
has
a chance to succeed without actually going into a blocking state.
In subsequent patches, we recognize that a syscall will block by
calling a
non-blocking poll (from inside the system call implementation) and
checking
for events. When events show up during the poll, it signifies that the
call
would not have blocked and the syscall is allowed to proceed (calling an
underlying host system call if necessary). If no events are returned
from the
poll, we generate the fault and try the instruction for the thread
context
at a distant tick. Note that retrying every tick is not efficient.
As an aside, the simulator has some multi-threading support for the
event
queue, but it is not used by default and needs work. Even if the event
queue
was completely multi-threaded, meaning that there is a hardware thread
on
the host servicing a single simulator thread contexts with a 1:1 mapping
between them, it's still possible to run into deadlock due to the event
queue
barriers on quantum boundaries. The solution of replaying at a later
tick
is the simplest solution and solves the problem generally.
diffstat:
src/arch/alpha/isa/decoder.isa | 2 +-
src/arch/arm/faults.cc | 3 ++-
src/arch/mips/isa/decoder.isa | 2 +-
src/arch/power/isa/decoder.isa | 2 +-
src/arch/riscv/faults.cc | 3 ++-
src/arch/sparc/faults.cc | 3 ++-
src/arch/sparc/linux/process.cc | 14 ++++++++------
src/arch/sparc/linux/process.hh | 4 ++--
src/arch/sparc/process.cc | 2 +-
src/arch/sparc/process.hh | 2 +-
src/arch/x86/isa/decoder/one_byte_opcodes.isa | 5 +++--
src/arch/x86/isa/decoder/two_byte_opcodes.isa | 10 ++++++----
src/arch/x86/process.cc | 4 ++--
src/arch/x86/process.hh | 2 +-
src/arch/x86/pseudo_inst.cc | 4 +++-
src/cpu/BaseCPU.py | 2 ++
src/cpu/base.cc | 3 ++-
src/cpu/base.hh | 2 ++
src/cpu/checker/cpu.hh | 2 +-
src/cpu/checker/thread_context.hh | 4 ++--
src/cpu/exec_context.hh | 2 +-
src/cpu/minor/exec_context.hh | 4 ++--
src/cpu/o3/commit.hh | 2 +-
src/cpu/o3/commit_impl.hh | 14 +++++++++-----
src/cpu/o3/cpu.cc | 4 ++--
src/cpu/o3/cpu.hh | 2 +-
src/cpu/o3/dyn_inst.hh | 2 +-
src/cpu/o3/dyn_inst_impl.hh | 4 ++--
src/cpu/o3/thread_context.hh | 4 ++--
src/cpu/o3/thread_state.hh | 5 ++++-
src/cpu/simple/atomic.cc | 9 ++++++++-
src/cpu/simple/exec_context.hh | 4 ++--
src/cpu/simple/timing.cc | 10 ++++++++--
src/cpu/simple_thread.hh | 4 ++--
src/cpu/thread_context.hh | 6 +++---
src/sim/faults.cc | 5 +++++
src/sim/faults.hh | 16 ++++++++++++++++
src/sim/process.cc | 4 ++--
src/sim/process.hh | 2 +-
src/sim/syscall_desc.cc | 12 +++++++++---
src/sim/syscall_desc.hh | 5 ++++-
41 files changed, 130 insertions(+), 65 deletions(-)
diffs (truncated from 738 to 300 lines):
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/alpha/isa/decoder.isa
--- a/src/arch/alpha/isa/decoder.isa Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/alpha/isa/decoder.isa Mon Jul 20 09:15:21 2015 -0500
@@ -840,7 +840,7 @@
exitSimLoop("halt instruction encountered");
}}, IsNonSpeculative);
0x83: callsys({{
- xc->syscall(R0);
+ xc->syscall(R0, &fault);
}}, IsSerializeAfter, IsNonSpeculative, IsSyscall);
// Read uniq reg into ABI return value register (r0)
0x9e: rduniq({{ R0 = Runiq; }}, IsIprAccess);
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/arm/faults.cc
--- a/src/arch/arm/faults.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/arm/faults.cc Mon Jul 20 09:15:21 2015 -0500
@@ -784,7 +784,8 @@
callNum = tc->readIntReg(INTREG_X8);
else
callNum = tc->readIntReg(INTREG_R7);
- tc->syscall(callNum);
+ Fault fault;
+ tc->syscall(callNum, &fault);
// Advance the PC since that won't happen automatically.
PCState pc = tc->pcState();
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/mips/isa/decoder.isa
--- a/src/arch/mips/isa/decoder.isa Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/mips/isa/decoder.isa Mon Jul 20 09:15:21 2015 -0500
@@ -164,7 +164,7 @@
0x2: movz({{ Rd = (Rt == 0) ? Rs : Rd; }});
0x3: movn({{ Rd = (Rt != 0) ? Rs : Rd; }});
0x4: decode FullSystemInt {
- 0: syscall_se({{ xc->syscall(R2); }},
+ 0: syscall_se({{ xc->syscall(R2, &fault); }},
IsSerializeAfter, IsNonSpeculative);
default: syscall({{ fault =
std::make_shared<SystemCallFault>(); }});
}
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/power/isa/decoder.isa
--- a/src/arch/power/isa/decoder.isa Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/power/isa/decoder.isa Mon Jul 20 09:15:21 2015 -0500
@@ -512,7 +512,7 @@
55: stfdu({{ Mem_df = Fs; }});
}
- 17: IntOp::sc({{ xc->syscall(R0); }},
+ 17: IntOp::sc({{ xc->syscall(R0, &fault); }},
[ IsSyscall, IsNonSpeculative, IsSerializeAfter ]);
format FloatArithOp {
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/riscv/faults.cc
--- a/src/arch/riscv/faults.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/riscv/faults.cc Mon Jul 20 09:15:21 2015 -0500
@@ -87,5 +87,6 @@
void
SyscallFault::invoke_se(ThreadContext *tc, const StaticInstPtr &inst)
{
- tc->syscall(tc->readIntReg(SyscallNumReg));
+ Fault *fault = NoFault;
+ tc->syscall(tc->readIntReg(SyscallNumReg), fault);
}
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/sparc/faults.cc
--- a/src/arch/sparc/faults.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/sparc/faults.cc Mon Jul 20 09:15:21 2015 -0500
@@ -811,7 +811,8 @@
SparcProcess *sp = dynamic_cast<SparcProcess *>(p);
assert(sp);
- sp->handleTrap(_n, tc);
+ Fault fault;
+ sp->handleTrap(_n, tc, &fault);
// We need to explicitly advance the pc, since that's not done for us
// on a faulting instruction
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/sparc/linux/process.cc
--- a/src/arch/sparc/linux/process.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/sparc/linux/process.cc Mon Jul 20 09:15:21 2015 -0500
@@ -65,14 +65,15 @@
: Sparc32Process(params, objFile)
{}
-void Sparc32LinuxProcess::handleTrap(int trapNum, ThreadContext *tc)
+void Sparc32LinuxProcess::handleTrap(int trapNum, ThreadContext *tc,
+ Fault *fault)
{
switch (trapNum) {
case 0x10: //Linux 32 bit syscall trap
- tc->syscall(tc->readIntReg(1));
+ tc->syscall(tc->readIntReg(1), fault);
break;
default:
- SparcProcess::handleTrap(trapNum, tc);
+ SparcProcess::handleTrap(trapNum, tc, fault);
}
}
@@ -81,14 +82,15 @@
: Sparc64Process(params, objFile)
{}
-void Sparc64LinuxProcess::handleTrap(int trapNum, ThreadContext *tc)
+void Sparc64LinuxProcess::handleTrap(int trapNum, ThreadContext *tc,
+ Fault *fault)
{
switch (trapNum) {
// case 0x10: // Linux 32 bit syscall trap
case 0x6d: // Linux 64 bit syscall trap
- tc->syscall(tc->readIntReg(1));
+ tc->syscall(tc->readIntReg(1), fault);
break;
default:
- SparcProcess::handleTrap(trapNum, tc);
+ SparcProcess::handleTrap(trapNum, tc, fault);
}
}
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/sparc/linux/process.hh
--- a/src/arch/sparc/linux/process.hh Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/sparc/linux/process.hh Mon Jul 20 09:15:21 2015 -0500
@@ -70,7 +70,7 @@
return SparcLinuxProcess::getDesc32(callnum);
}
- void handleTrap(int trapNum, ThreadContext *tc);
+ void handleTrap(int trapNum, ThreadContext *tc, Fault *fault);
};
/// A process with emulated 32 bit SPARC/Linux syscalls.
@@ -86,7 +86,7 @@
return SparcLinuxProcess::getDesc(callnum);
}
- void handleTrap(int trapNum, ThreadContext *tc);
+ void handleTrap(int trapNum, ThreadContext *tc, Fault *fault);
};
SyscallReturn getresuidFunc(SyscallDesc *desc, int num,
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/sparc/process.cc
--- a/src/arch/sparc/process.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/sparc/process.cc Mon Jul 20 09:15:21 2015 -0500
@@ -71,7 +71,7 @@
}
void
-SparcProcess::handleTrap(int trapNum, ThreadContext *tc)
+SparcProcess::handleTrap(int trapNum, ThreadContext *tc, Fault *fault)
{
PCState pc = tc->pcState();
switch (trapNum) {
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/sparc/process.hh
--- a/src/arch/sparc/process.hh Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/sparc/process.hh Mon Jul 20 09:15:21 2015 -0500
@@ -61,7 +61,7 @@
public:
// Handles traps which request services from the operating system
- virtual void handleTrap(int trapNum, ThreadContext *tc);
+ virtual void handleTrap(int trapNum, ThreadContext *tc, Fault *fault);
Addr readFillStart() { return fillStart; }
Addr readSpillStart() { return spillStart; }
diff -r 39bff8406c3b -r 5ea85692a53e
src/arch/x86/isa/decoder/one_byte_opcodes.isa
--- a/src/arch/x86/isa/decoder/one_byte_opcodes.isa Mon Jul 20 09:15:21
2015 -0500
+++ b/src/arch/x86/isa/decoder/one_byte_opcodes.isa Mon Jul 20 09:15:21
2015 -0500
@@ -400,8 +400,9 @@
// will sign extend it, and there's no easy way to
// specify only checking the first byte.
0xffffffffffffff80:
- SyscallInst::int80('xc->syscall(Rax)',
- IsSyscall, IsNonSpeculative,
IsSerializeAfter);
+ SyscallInst::int80('xc->syscall(Rax, &fault)',
+ IsSyscall, IsNonSpeculative,
+ IsSerializeAfter);
}
default: Inst::INT(Ib);
diff -r 39bff8406c3b -r 5ea85692a53e
src/arch/x86/isa/decoder/two_byte_opcodes.isa
--- a/src/arch/x86/isa/decoder/two_byte_opcodes.isa Mon Jul 20 09:15:21
2015 -0500
+++ b/src/arch/x86/isa/decoder/two_byte_opcodes.isa Mon Jul 20 09:15:21
2015 -0500
@@ -235,8 +235,9 @@
}
}
0x05: decode FullSystemInt {
- 0: SyscallInst::syscall('xc->syscall(Rax)',
- IsSyscall, IsNonSpeculative, IsSerializeAfter);
+ 0: SyscallInst::syscall('xc->syscall(Rax, &fault)',
+ IsSyscall, IsNonSpeculative,
+ IsSerializeAfter);
default: decode MODE_MODE {
0x0: decode MODE_SUBMODE {
0x0: Inst::SYSCALL_64();
@@ -422,8 +423,9 @@
0x2: Inst::RDMSR();
0x3: rdpmc();
0x4: decode FullSystemInt {
- 0: SyscallInst::sysenter('xc->syscall(Rax)',
- IsSyscall, IsNonSpeculative, IsSerializeAfter);
+ 0: SyscallInst::sysenter('xc->syscall(Rax, &fault)',
+ IsSyscall, IsNonSpeculative,
+ IsSerializeAfter);
default: sysenter();
}
0x5: sysexit();
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/x86/process.cc
--- a/src/arch/x86/process.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/x86/process.cc Mon Jul 20 09:15:21 2015 -0500
@@ -134,7 +134,7 @@
}
void
-I386Process::syscall(int64_t callnum, ThreadContext *tc)
+I386Process::syscall(int64_t callnum, ThreadContext *tc, Fault *fault)
{
TheISA::PCState pc = tc->pcState();
Addr eip = pc.pc();
@@ -143,7 +143,7 @@
pc.npc(vsyscallPage.base + vsyscallPage.vsysexitOffset);
tc->pcState(pc);
}
- X86Process::syscall(callnum, tc);
+ X86Process::syscall(callnum, tc, fault);
}
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/x86/process.hh
--- a/src/arch/x86/process.hh Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/x86/process.hh Mon Jul 20 09:15:21 2015 -0500
@@ -130,7 +130,7 @@
void argsInit(int intSize, int pageSize);
void initState();
- void syscall(int64_t callnum, ThreadContext *tc);
+ void syscall(int64_t callnum, ThreadContext *tc, Fault *fault);
X86ISA::IntReg getSyscallArg(ThreadContext *tc, int &i);
X86ISA::IntReg getSyscallArg(ThreadContext *tc, int &i, int width);
void setSyscallArg(ThreadContext *tc, int i, X86ISA::IntReg val);
diff -r 39bff8406c3b -r 5ea85692a53e src/arch/x86/pseudo_inst.cc
--- a/src/arch/x86/pseudo_inst.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/arch/x86/pseudo_inst.cc Mon Jul 20 09:15:21 2015 -0500
@@ -49,7 +49,9 @@
{
DPRINTF(PseudoInst, "PseudoInst::m5Syscall()\n");
- tc->syscall(tc->readIntReg(INTREG_RAX));
+ Fault fault;
+ tc->syscall(tc->readIntReg(INTREG_RAX), &fault);
+
MiscReg rflags = tc->readMiscReg(MISCREG_RFLAGS);
rflags &= ~(1 << 16);
tc->setMiscReg(MISCREG_RFLAGS, rflags);
diff -r 39bff8406c3b -r 5ea85692a53e src/cpu/BaseCPU.py
--- a/src/cpu/BaseCPU.py Mon Jul 20 09:15:21 2015 -0500
+++ b/src/cpu/BaseCPU.py Mon Jul 20 09:15:21 2015 -0500
@@ -142,6 +142,8 @@
checker = Param.BaseCPU(NULL, "checker CPU")
+ syscallRetryLatency = Param.Cycles(10000, "Cycles to wait until retry")
+
do_checkpoint_insts = Param.Bool(True,
"enable checkpoint pseudo instructions")
do_statistics_insts = Param.Bool(True,
diff -r 39bff8406c3b -r 5ea85692a53e src/cpu/base.cc
--- a/src/cpu/base.cc Mon Jul 20 09:15:21 2015 -0500
+++ b/src/cpu/base.cc Mon Jul 20 09:15:21 2015 -0500
@@ -135,7 +135,8 @@
numThreads(p->numThreads), system(p->system),
functionTraceStream(nullptr), currentFunctionStart(0),
currentFunctionEnd(0), functionEntryTick(0),
- addressMonitor(p->numThreads)
+ addressMonitor(p->numThreads),
+ syscallRetryLatency(p->syscallRetryLatency)
{
// if Python did not provide a valid ID, do it here
if (_cpuId == -1 ) {
diff -r 39bff8406c3b -r 5ea85692a53e src/cpu/base.hh
--- a/src/cpu/base.hh Mon Jul 20 09:15:21 2015 -0500
+++ b/src/cpu/base.hh Mon Jul 20 09:15:21 2015 -0500
@@ -588,6 +588,8 @@
assert(tid < numThreads);
return &addressMonitor[tid];
}
+
+ Cycles syscallRetryLatency;
};
#endif // THE_ISA == NULL_ISA
diff -r 39bff8406c3b -r 5ea85692a53e src/cpu/checker/cpu.hh
--- a/src/cpu/checker/cpu.hh Mon Jul 20 09:15:21 2015 -0500
+++ b/src/cpu/checker/cpu.hh Mon Jul 20 09:15:21 2015 -0500
@@ -393,7 +393,7 @@
void wakeup(ThreadID tid) override { }
// Assume that the normal CPU's call to syscall was successful.
// The checker's state would have already been updated by the syscall.
- void syscall(int64_t callnum) override { }
+ void syscall(int64_t callnum, Fault *fault) override { }
void handleError()
{
diff -r 39bff8406c3b -r 5ea85692a53e src/cpu/checker/thread_context.hh
--- a/src/cpu/checker/thread_context.hh Mon Jul 20 09:15:21 2015 -0500
+++ b/src/cpu/checker/thread_context.hh Mon Jul 20 09:15:21 2015 -0500
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
