Diff
Modified: trunk/Source/_javascript_Core/CMakeLists.txt (277382 => 277383)
--- trunk/Source/_javascript_Core/CMakeLists.txt 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/CMakeLists.txt 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1063,6 +1063,7 @@
runtime/ScriptExecutable.h
runtime/ScriptFetchParameters.h
runtime/ScriptFetcher.h
+ runtime/SlowPathFunction.h
runtime/SlowPathReturnType.h
runtime/SmallStrings.h
runtime/SourceOrigin.h
Modified: trunk/Source/_javascript_Core/ChangeLog (277382 => 277383)
--- trunk/Source/_javascript_Core/ChangeLog 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/ChangeLog 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,3 +1,113 @@
+2021-05-12 Mark Lam <[email protected]>
+
+ Implement some common Baseline JIT slow paths using JIT thunks.
+ https://bugs.webkit.org/show_bug.cgi?id=225682
+
+ Reviewed by Filip Pizlo.
+
+ This patch implements the following changes:
+
+ 1. Implement exception handling thunks:
+ a. handleExceptionGenerator, which calls operationLookupExceptionHandler().
+ b. handleExceptionWithCallFrameRollbackGenerator, which calls
+ operationLookupExceptionHandlerFromCallerFrame().
+
+ All the JIT tiers were emitting their own copy of these routines to call these
+ operation, one per CodeBlock. We now emit 2 thunks for these and have all the
+ tiers just jump to them.
+
+ PolymorphicAccess also now uses the handleExceptionGenerator thunk.
+
+ DFG::JITCompiler::compileExceptionHandlers() has one small behavior difference
+ before it calls operationLookupExceptionHandlerFromCallerFrame(): it first
+ re-sets the top of stack for the function where we are about to throw a
+ StackOverflowError from. This re-setting of top of stack is useless because
+ we're imminently unwinding out of at least this frame for the StackOverflowError.
+ Hence, it is ok to use the handleExceptionWithCallFrameRollbackGenerator thunk
+ here as well. Note that no other tiers does this re-setting of top of stack.
+
+ FTLLowerDFGToB3 has one case using operationLookupExceptionHandlerFromCallerFrame()
+ which cannot be refactored to use these thunks because it does additional
+ work to throw a StackOverflowError. A different thunk will be needed. I left
+ it alone for now.
+
+ 2. Introduce JITThunks::existingCTIStub(ThunkGenerator, NoLockingNecessaryTag) so
+ that a thunk can get a pointer to another thunk without locking the JITThunks
+ lock. Otherwise, deadlock ensues.
+
+ 3. Change SlowPathCall to emit and use thunks instead of emitting a blob of code
+ to call a slow path function for every bytecode in a CodeBlock.
+
+ 4. Introduce JITThunks::ctiSlowPathFunctionStub() to manage these SlowPathFunction
+ thunks.
+
+ 5. Introduce JITThunks::preinitializeAggressiveCTIThunks() to initialize these
+ thunks at VM initialization time. Pre-initializing them has multiple benefits:
+ a. the thunks are not scattered through out JIT memory, thereby reducing
+ fragmentation.
+ b. we don't spend time at runtime compiling them when the user is interacting
+ with the VM. Conceptually, these thunks can be VM independent and can be
+ shared by VMs process-wide. However, it will require some additional work.
+ For now, the thunks remain bound to a specific VM instance.
+
+ These changes are only enabled when ENABLE(EXTRA_CTI_THUNKS), which is currently
+ only available for ARM64 and non-Windows x86_64.
+
+ This patch has passed JSC tests on AS Mac.
+
+ With this patch, --dumpLinkBufferStats shows the following changes in emitted
+ JIT code size (using a single run of the CLI version of JetStream2 on AS Mac):
+
+ Base New Diff
+
+ BaselineJIT: 89089964 (84.962811 MB) 84624776 (80.704475 MB) 0.95x (reduction)
+ DFG: 39117360 (37.305222 MB) 36415264 (34.728302 MB) 0.93x (reduction)
+ Thunk: 23230968 (22.154778 MB) 23130336 (22.058807 MB) 1.00x
+ InlineCache: 22027416 (21.006981 MB) 21969728 (20.951965 MB) 1.00x
+ FTL: 6575772 (6.271145 MB) 6097336 (5.814873 MB) 0.93x (reduction)
+ Wasm: 2302724 (2.196049 MB) 2301956 (2.195316 MB) 1.00x
+ YarrJIT: 1538956 (1.467663 MB) 1522488 (1.451958 MB) 0.99x
+ CSSJIT: 0 0
+ Uncategorized: 0 0
+
+ * CMakeLists.txt:
+ * _javascript_Core.xcodeproj/project.pbxproj:
+ * Sources.txt:
+ * bytecode/CodeBlock.h:
+ (JSC::CodeBlock::offsetOfInstructionsRawPointer):
+ * bytecode/PolymorphicAccess.cpp:
+ (JSC::AccessGenerationState::emitExplicitExceptionHandler):
+ * dfg/DFGJITCompiler.cpp:
+ (JSC::DFG::JITCompiler::compileExceptionHandlers):
+ (JSC::DFG::JITCompiler::link):
+ * dfg/DFGJITCompiler.h:
+ * ftl/FTLCompile.cpp:
+ (JSC::FTL::compile):
+ * ftl/FTLLink.cpp:
+ (JSC::FTL::link):
+ * jit/JIT.cpp:
+ (JSC::JIT::link):
+ (JSC::JIT::privateCompileExceptionHandlers):
+ * jit/JIT.h:
+ * jit/JITThunks.cpp:
+ (JSC::JITThunks::existingCTIStub):
+ (JSC::JITThunks::ctiSlowPathFunctionStub):
+ (JSC::JITThunks::preinitializeExtraCTIThunks):
+ * jit/JITThunks.h:
+ * jit/SlowPathCall.cpp: Added.
+ (JSC::JITSlowPathCall::call):
+ (JSC::JITSlowPathCall::generateThunk):
+ * jit/SlowPathCall.h:
+ * jit/ThunkGenerators.cpp:
+ (JSC::handleExceptionGenerator):
+ (JSC::handleExceptionWithCallFrameRollbackGenerator):
+ (JSC::popThunkStackPreservesAndHandleExceptionGenerator):
+ * jit/ThunkGenerators.h:
+ * runtime/CommonSlowPaths.h:
+ * runtime/SlowPathFunction.h: Added.
+ * runtime/VM.cpp:
+ (JSC::VM::VM):
+
2021-05-12 Commit Queue <[email protected]>
Unreviewed, reverting r277346.
Modified: trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj (277382 => 277383)
--- trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1988,6 +1988,7 @@
FE7497E6209001B10003565B /* JSCPtrTag.h in Headers */ = {isa = PBXBuildFile; fileRef = FE7497E5209001B00003565B /* JSCPtrTag.h */; settings = {ATTRIBUTES = (Private, ); }; };
FE7C41961B97FC4B00F4D598 /* PingPongStackOverflowTest.cpp in Sources */ = {isa = PBXBuildFile; fileRef = FEDA50D41B97F442009A3B4F /* PingPongStackOverflowTest.cpp */; };
FE80C1971D775CDD008510C0 /* CatchScope.h in Headers */ = {isa = PBXBuildFile; fileRef = FE80C1961D775B27008510C0 /* CatchScope.h */; settings = {ATTRIBUTES = (Private, ); }; };
+ FE8C0312264A6911001A44AD /* SlowPathFunction.h in Headers */ = {isa = PBXBuildFile; fileRef = FE8C0311264A6910001A44AD /* SlowPathFunction.h */; settings = {ATTRIBUTES = (Private, ); }; };
FE8DE54B23AC1DAD005C9142 /* CacheableIdentifier.h in Headers */ = {isa = PBXBuildFile; fileRef = FE8DE54A23AC1DAD005C9142 /* CacheableIdentifier.h */; };
FE8DE54D23AC1E86005C9142 /* CacheableIdentifierInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = FE8DE54C23AC1E86005C9142 /* CacheableIdentifierInlines.h */; };
FE912B4F2531193300FABDDF /* AbstractSlotVisitor.h in Headers */ = {isa = PBXBuildFile; fileRef = FE912B4E2531193300FABDDF /* AbstractSlotVisitor.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -4010,6 +4011,7 @@
6A38CFA71E32B58B0060206F /* AsyncStackTrace.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AsyncStackTrace.cpp; sourceTree = "<group>"; };
6A38CFA81E32B58B0060206F /* AsyncStackTrace.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AsyncStackTrace.h; sourceTree = "<group>"; };
6AD2CB4C19B9140100065719 /* DebuggerEvalEnabler.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DebuggerEvalEnabler.h; sourceTree = "<group>"; };
+ 6B731CC02647A8370014646F /* SlowPathCall.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = SlowPathCall.cpp; sourceTree = "<group>"; };
6BA93C9590484C5BAD9316EA /* JSScriptFetcher.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSScriptFetcher.h; sourceTree = "<group>"; };
6BCCEC0325D1FA27000F391D /* VerifierSlotVisitorInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VerifierSlotVisitorInlines.h; sourceTree = "<group>"; };
70113D491A8DB093003848C4 /* IteratorOperations.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = IteratorOperations.cpp; sourceTree = "<group>"; };
@@ -5362,6 +5364,7 @@
FE80C1961D775B27008510C0 /* CatchScope.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CatchScope.h; sourceTree = "<group>"; };
FE80C1981D775FB4008510C0 /* CatchScope.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CatchScope.cpp; sourceTree = "<group>"; };
FE80C19A1D7768FD008510C0 /* ExceptionEventLocation.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ExceptionEventLocation.cpp; sourceTree = "<group>"; };
+ FE8C0311264A6910001A44AD /* SlowPathFunction.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SlowPathFunction.h; sourceTree = "<group>"; };
FE8DE54A23AC1DAD005C9142 /* CacheableIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CacheableIdentifier.h; sourceTree = "<group>"; };
FE8DE54C23AC1E86005C9142 /* CacheableIdentifierInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CacheableIdentifierInlines.h; sourceTree = "<group>"; };
FE8DE54E23AC3A8E005C9142 /* CacheableIdentifier.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = CacheableIdentifier.cpp; sourceTree = "<group>"; };
@@ -6302,6 +6305,7 @@
0F24E54B17EE274900ABB217 /* ScratchRegisterAllocator.h */,
0FEE98421A89227500754E93 /* SetupVarargsFrame.cpp */,
0FEE98401A8865B600754E93 /* SetupVarargsFrame.h */,
+ 6B731CC02647A8370014646F /* SlowPathCall.cpp */,
A709F2EF17A0AC0400512E98 /* SlowPathCall.h */,
E3F23A7B1ECF13E500978D99 /* Snippet.h */,
FE187A0C1C02EBA70038BBCA /* SnippetOperand.h */,
@@ -7836,6 +7840,7 @@
A7299DA017D12848005F5FF9 /* SetPrototype.h */,
0F2B66D617B6B5AB00A7AE3F /* SimpleTypedArrayController.cpp */,
0F2B66D717B6B5AB00A7AE3F /* SimpleTypedArrayController.h */,
+ FE8C0311264A6910001A44AD /* SlowPathFunction.h */,
0F5B4A321C84F0D600F1B17E /* SlowPathReturnType.h */,
93303FE80E6A72B500786E6A /* SmallStrings.cpp */,
93303FEA0E6A72C000786E6A /* SmallStrings.h */,
@@ -10242,6 +10247,7 @@
86E3C612167BABD7006D760A /* JSValue.h in Headers */,
86E3C61B167BABEE006D760A /* JSValueInternal.h in Headers */,
53E1F8F82154715A0001DDBC /* JSValuePrivate.h in Headers */,
+ FE8C0312264A6911001A44AD /* SlowPathFunction.h in Headers */,
BC18C42C0E16F5CD00B34460 /* JSValueRef.h in Headers */,
86E3C615167BABD7006D760A /* JSVirtualMachine.h in Headers */,
86E3C61D167BABEE006D760A /* JSVirtualMachineInternal.h in Headers */,
Modified: trunk/Source/_javascript_Core/Sources.txt (277382 => 277383)
--- trunk/Source/_javascript_Core/Sources.txt 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/Sources.txt 2021-05-12 19:11:25 UTC (rev 277383)
@@ -676,6 +676,7 @@
jit/Repatch.cpp
jit/ScratchRegisterAllocator.cpp
jit/SetupVarargsFrame.cpp
+jit/SlowPathCall.cpp
jit/TagRegistersMode.cpp
jit/TempRegisterSet.cpp
jit/ThunkGenerators.cpp
Modified: trunk/Source/_javascript_Core/bytecode/CodeBlock.h (277382 => 277383)
--- trunk/Source/_javascript_Core/bytecode/CodeBlock.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/bytecode/CodeBlock.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -915,6 +915,8 @@
MetadataTable* metadataTable() { return m_metadata.get(); }
const void* instructionsRawPointer() { return m_instructionsRawPointer; }
+ static ptrdiff_t offsetOfInstructionsRawPointer() { return OBJECT_OFFSETOF(CodeBlock, m_instructionsRawPointer); }
+
bool loopHintsAreEligibleForFuzzingEarlyReturn()
{
// Some builtins are required to always complete the loops they run.
Modified: trunk/Source/_javascript_Core/bytecode/PolymorphicAccess.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/bytecode/PolymorphicAccess.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/bytecode/PolymorphicAccess.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -40,6 +40,7 @@
#include "StructureStubClearingWatchpoint.h"
#include "StructureStubInfo.h"
#include "SuperSampler.h"
+#include "ThunkGenerators.h"
#include <wtf/CommaPrinter.h>
#include <wtf/ListDump.h>
@@ -226,6 +227,14 @@
linkBuffer.link(jumpToOSRExitExceptionHandler, originalHandler.nativeCode);
});
} else {
+#if ENABLE(EXTRA_CTI_THUNKS)
+ CCallHelpers::Jump jumpToExceptionHandler = jit->jump();
+ VM* vm = &m_vm;
+ jit->addLinkTask(
+ [=] (LinkBuffer& linkBuffer) {
+ linkBuffer.link(jumpToExceptionHandler, CodeLocationLabel(vm->getCTIStub(handleExceptionGenerator).retaggedCode<NoPtrTag>()));
+ });
+#else
jit->setupArguments<decltype(operationLookupExceptionHandler)>(CCallHelpers::TrustedImmPtr(&m_vm));
jit->prepareCallOperation(m_vm);
CCallHelpers::Call lookupExceptionHandlerCall = jit->call(OperationPtrTag);
@@ -234,6 +243,7 @@
linkBuffer.link(lookupExceptionHandlerCall, FunctionPtr<OperationPtrTag>(operationLookupExceptionHandler));
});
jit->jumpToExceptionHandler(m_vm);
+#endif
}
}
Modified: trunk/Source/_javascript_Core/dfg/DFGJITCompiler.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/dfg/DFGJITCompiler.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/dfg/DFGJITCompiler.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -137,6 +137,7 @@
void JITCompiler::compileExceptionHandlers()
{
+#if !ENABLE(EXTRA_CTI_THUNKS)
if (!m_exceptionChecksWithCallFrameRollback.empty()) {
m_exceptionChecksWithCallFrameRollback.link(this);
@@ -165,6 +166,7 @@
jumpToExceptionHandler(vm());
}
+#endif // ENABLE(EXTRA_CTI_THUNKS)
}
void JITCompiler::link(LinkBuffer& linkBuffer)
@@ -281,7 +283,14 @@
linkBuffer.locationOf<JSInternalPtrTag>(record.slowPath),
linkBuffer.locationOfNearCall<JSInternalPtrTag>(record.call));
}
-
+
+#if ENABLE(EXTRA_CTI_THUNKS)
+ if (!m_exceptionChecks.empty())
+ linkBuffer.link(m_exceptionChecks, CodeLocationLabel(vm().getCTIStub(handleExceptionGenerator).retaggedCode<NoPtrTag>()));
+ if (!m_exceptionChecksWithCallFrameRollback.empty())
+ linkBuffer.link(m_exceptionChecksWithCallFrameRollback, CodeLocationLabel(vm().getCTIStub(handleExceptionWithCallFrameRollbackGenerator).retaggedCode<NoPtrTag>()));
+#endif // ENABLE(EXTRA_CTI_THUNKS)
+
MacroAssemblerCodeRef<JITThunkPtrTag> osrExitThunk = vm().getCTIStub(osrExitGenerationThunkGenerator);
auto target = CodeLocationLabel<JITThunkPtrTag>(osrExitThunk.code());
for (unsigned i = 0; i < m_osrExit.size(); ++i) {
Modified: trunk/Source/_javascript_Core/dfg/DFGJITCompiler.h (277382 => 277383)
--- trunk/Source/_javascript_Core/dfg/DFGJITCompiler.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/dfg/DFGJITCompiler.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -323,7 +323,7 @@
Vector<CallLinkRecord> m_calls;
JumpList m_exceptionChecks;
JumpList m_exceptionChecksWithCallFrameRollback;
-
+
Vector<Label> m_blockHeads;
Modified: trunk/Source/_javascript_Core/ftl/FTLCompile.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/ftl/FTLCompile.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/ftl/FTLCompile.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -40,6 +40,7 @@
#include "FTLJITCode.h"
#include "LinkBuffer.h"
#include "PCToCodeOriginMap.h"
+#include "ThunkGenerators.h"
#include <wtf/RecursableLambda.h>
namespace JSC { namespace FTL {
@@ -125,6 +126,14 @@
// Emit the exception handler.
*state.exceptionHandler = jit.label();
+#if ENABLE(EXTRA_CTI_THUNKS)
+ CCallHelpers::Jump handler = jit.jump();
+ VM* vmPtr = &vm;
+ jit.addLinkTask(
+ [=] (LinkBuffer& linkBuffer) {
+ linkBuffer.link(handler, CodeLocationLabel(vmPtr->getCTIStub(handleExceptionGenerator).retaggedCode<NoPtrTag>()));
+ });
+#else
jit.copyCalleeSavesToEntryFrameCalleeSavesBuffer(vm.topEntryFrame);
jit.move(MacroAssembler::TrustedImmPtr(&vm), GPRInfo::argumentGPR0);
jit.prepareCallOperation(vm);
@@ -134,6 +143,7 @@
[=] (LinkBuffer& linkBuffer) {
linkBuffer.link(call, FunctionPtr<OperationPtrTag>(operationLookupExceptionHandler));
});
+#endif // ENABLE(EXTRA_CTI_THUNKS)
state.finalizer->b3CodeLinkBuffer = makeUnique<LinkBuffer>(jit, codeBlock, LinkBuffer::Profile::FTL, JITCompilationCanFail);
Modified: trunk/Source/_javascript_Core/ftl/FTLLink.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/ftl/FTLLink.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/ftl/FTLLink.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -145,6 +145,9 @@
jit.storePtr(GPRInfo::callFrameRegister, &vm.topCallFrame);
CCallHelpers::Call callArityCheck = jit.call(OperationPtrTag);
+#if ENABLE(EXTRA_CTI_THUNKS)
+ auto jumpToExceptionHandler = jit.branch32(CCallHelpers::LessThan, GPRInfo::returnValueGPR, CCallHelpers::TrustedImm32(0));
+#else
auto noException = jit.branch32(CCallHelpers::GreaterThanOrEqual, GPRInfo::returnValueGPR, CCallHelpers::TrustedImm32(0));
jit.copyCalleeSavesToEntryFrameCalleeSavesBuffer(vm.topEntryFrame);
jit.move(CCallHelpers::TrustedImmPtr(&vm), GPRInfo::argumentGPR0);
@@ -152,6 +155,7 @@
CCallHelpers::Call callLookupExceptionHandlerFromCallerFrame = jit.call(OperationPtrTag);
jit.jumpToExceptionHandler(vm);
noException.link(&jit);
+#endif // ENABLE(EXTRA_CTI_THUNKS)
if (ASSERT_ENABLED) {
jit.load64(vm.addressOfException(), GPRInfo::regT1);
@@ -174,7 +178,11 @@
return;
}
linkBuffer->link(callArityCheck, FunctionPtr<OperationPtrTag>(codeBlock->isConstructor() ? operationConstructArityCheck : operationCallArityCheck));
+#if ENABLE(EXTRA_CTI_THUNKS)
+ linkBuffer->link(jumpToExceptionHandler, CodeLocationLabel(vm.getCTIStub(handleExceptionWithCallFrameRollbackGenerator).retaggedCode<NoPtrTag>()));
+#else
linkBuffer->link(callLookupExceptionHandlerFromCallerFrame, FunctionPtr<OperationPtrTag>(operationLookupExceptionHandlerFromCallerFrame));
+#endif
linkBuffer->link(callArityFixup, FunctionPtr<JITThunkPtrTag>(vm.getCTIStub(arityFixupGenerator).code()));
linkBuffer->link(mainPathJumps, state.generatedFunction);
}
Modified: trunk/Source/_javascript_Core/jit/JIT.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/jit/JIT.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/jit/JIT.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -897,6 +897,12 @@
handler.nativeCode = patchBuffer.locationOf<ExceptionHandlerPtrTag>(m_labels[handler.target]);
}
+#if ENABLE(EXTRA_CTI_THUNKS)
+ if (!m_exceptionChecks.empty())
+ patchBuffer.link(m_exceptionChecks, CodeLocationLabel(vm().getCTIStub(handleExceptionGenerator).retaggedCode<NoPtrTag>()));
+ if (!m_exceptionChecksWithCallFrameRollback.empty())
+ patchBuffer.link(m_exceptionChecksWithCallFrameRollback, CodeLocationLabel(vm().getCTIStub(handleExceptionWithCallFrameRollbackGenerator).retaggedCode<NoPtrTag>()));
+#endif
for (auto& record : m_nearCalls) {
if (record.callee)
@@ -906,7 +912,7 @@
if (record.callee)
patchBuffer.link(record.from, record.callee);
}
-
+
finalizeInlineCaches(m_getByIds, patchBuffer);
finalizeInlineCaches(m_getByVals, patchBuffer);
finalizeInlineCaches(m_getByIdsWithThis, patchBuffer);
@@ -918,7 +924,11 @@
finalizeInlineCaches(m_privateBrandAccesses, patchBuffer);
if (m_byValCompilationInfo.size()) {
+#if ENABLE(EXTRA_CTI_THUNKS)
+ CodeLocationLabel exceptionHandler(vm().getCTIStub(handleExceptionGenerator).retaggedCode<ExceptionHandlerPtrTag>());
+#else
CodeLocationLabel<ExceptionHandlerPtrTag> exceptionHandler = patchBuffer.locationOf<ExceptionHandlerPtrTag>(m_exceptionHandler);
+#endif
for (const auto& byValCompilationInfo : m_byValCompilationInfo) {
PatchableJump patchableNotIndexJump = byValCompilationInfo.notIndexJump;
@@ -1009,6 +1019,7 @@
void JIT::privateCompileExceptionHandlers()
{
+#if !ENABLE(EXTRA_CTI_THUNKS)
if (!m_exceptionChecksWithCallFrameRollback.empty()) {
m_exceptionChecksWithCallFrameRollback.link(this);
@@ -1033,6 +1044,7 @@
m_farCalls.append(FarCallRecord(call(OperationPtrTag), FunctionPtr<OperationPtrTag>(operationLookupExceptionHandler)));
jumpToExceptionHandler(vm());
}
+#endif // ENABLE(EXTRA_CTI_THUNKS)
}
void JIT::doMainThreadPreparationBeforeCompile()
Modified: trunk/Source/_javascript_Core/jit/JIT.h (277382 => 277383)
--- trunk/Source/_javascript_Core/jit/JIT.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/jit/JIT.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -987,7 +987,9 @@
JumpList m_exceptionChecks;
JumpList m_exceptionChecksWithCallFrameRollback;
+#if !ENABLE(EXTRA_CTI_THUNKS)
Label m_exceptionHandler;
+#endif
unsigned m_getByIdIndex { UINT_MAX };
unsigned m_getByValIndex { UINT_MAX };
Modified: trunk/Source/_javascript_Core/jit/JITThunks.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/jit/JITThunks.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/jit/JITThunks.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2012-2020 Apple Inc. All rights reserved.
+ * Copyright (C) 2012-2021 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -28,8 +28,10 @@
#if ENABLE(JIT)
+#include "CommonSlowPaths.h"
#include "JITCode.h"
#include "JSCJSValueInlines.h"
+#include "SlowPathCall.h"
#include "ThunkGenerators.h"
#include "VM.h"
@@ -141,6 +143,11 @@
MacroAssemblerCodeRef<JITThunkPtrTag> JITThunks::existingCTIStub(ThunkGenerator generator)
{
LockHolder locker(m_lock);
+ return existingCTIStub(generator, NoLockingNecessary);
+}
+
+MacroAssemblerCodeRef<JITThunkPtrTag> JITThunks::existingCTIStub(ThunkGenerator generator, NoLockingNecessaryTag)
+{
CTIStubMap::iterator entry = m_ctiStubMap.find(generator);
if (entry == m_ctiStubMap.end())
return MacroAssemblerCodeRef<JITThunkPtrTag>();
@@ -147,6 +154,23 @@
return entry->value;
}
+#if ENABLE(EXTRA_CTI_THUNKS)
+
+MacroAssemblerCodeRef<JITThunkPtrTag> JITThunks::ctiSlowPathFunctionStub(VM& vm, SlowPathFunction slowPathFunction)
+{
+ LockHolder locker(m_lock);
+ auto key = bitwise_cast<ThunkGenerator>(slowPathFunction);
+ CTIStubMap::AddResult entry = m_ctiStubMap.add(key, MacroAssemblerCodeRef<JITThunkPtrTag>());
+ if (entry.isNewEntry) {
+ // Compilation thread can only retrieve existing entries.
+ ASSERT(!isCompilationThread());
+ entry.iterator->value = JITSlowPathCall::generateThunk(vm, slowPathFunction);
+ }
+ return entry.iterator->value;
+}
+
+#endif // ENABLE(EXTRA_CTI_THUNKS)
+
struct JITThunks::HostKeySearcher {
static unsigned hash(const HostFunctionKey& key) { return WeakNativeExecutableHash::hash(key); }
static bool equal(const Weak<NativeExecutable>& a, const HostFunctionKey& b) { return WeakNativeExecutableHash::equal(a, b); }
@@ -233,6 +257,100 @@
return hostFunctionStub(vm, function, callHostFunctionAsConstructor, generator, intrinsic, nullptr, name);
}
+#if ENABLE(EXTRA_CTI_THUNKS)
+
+void JITThunks::preinitializeExtraCTIThunks(VM& vm)
+{
+ if (!Options::useJIT())
+ return;
+
+ // These 3 should always be initialized first in the following order because
+ // the other thunk generators rely on these already being initialized.
+ ctiStub(vm, handleExceptionGenerator);
+ ctiStub(vm, handleExceptionWithCallFrameRollbackGenerator);
+ ctiStub(vm, popThunkStackPreservesAndHandleExceptionGenerator);
+
+#define INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(name) ctiSlowPathFunctionStub(vm, slow_path_##name)
+
+ // From the BaselineJIT DEFINE_SLOW_OP list:
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(in_by_val);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(less);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(lesseq);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(greater);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(greatereq);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(is_callable);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(is_constructor);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(typeof);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(typeof_is_object);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(typeof_is_function);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(strcat);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(push_with_scope);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_lexical_environment);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(get_by_val_with_this);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(put_by_id_with_this);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(put_by_val_with_this);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(resolve_scope_for_hoisting_func_decl_in_eval);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(define_data_property);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(define_accessor_property);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(unreachable);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(throw_static_error);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(new_array_with_spread);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(new_array_buffer);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(spread);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(get_enumerable_length);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(has_enumerable_property);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(get_property_enumerator);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_index_string);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_direct_arguments);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_scoped_arguments);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_cloned_arguments);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_arguments_butterfly);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_rest);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_promise);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(new_promise);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_generator);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_async_generator);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(new_generator);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(pow);
+
+ // From the BaselineJIT DEFINE_SLOWCASE_SLOW_OP list:
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(unsigned);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(inc);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(dec);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(bitnot);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(bitand);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(bitor);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(bitxor);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(lshift);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(rshift);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(urshift);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(div);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_this);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_promise);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_generator);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(create_async_generator);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_this);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_primitive);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_number);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_numeric);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_string);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_object);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(not);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(stricteq);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(nstricteq);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(get_direct_pname);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(get_prototype_of);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(has_enumerable_structure_property);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(has_own_structure_property);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(in_structure_property);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(resolve_scope);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(check_tdz);
+ INIT_BASELINE_SLOW_PATH_CALL_ROUTINE(to_property_key);
+#undef INIT_BASELINE_ROUTINE
+}
+
+#endif // ENABLE(EXTRA_CTI_THUNKS)
+
} // namespace JSC
#endif // ENABLE(JIT)
Modified: trunk/Source/_javascript_Core/jit/JITThunks.h (277382 => 277383)
--- trunk/Source/_javascript_Core/jit/JITThunks.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/jit/JITThunks.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2012-2019 Apple Inc. All rights reserved.
+ * Copyright (C) 2012-2021 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -30,6 +30,7 @@
#include "CallData.h"
#include "Intrinsic.h"
#include "MacroAssemblerCodeRef.h"
+#include "SlowPathFunction.h"
#include "ThunkGenerator.h"
#include "Weak.h"
#include "WeakHandleOwner.h"
@@ -52,6 +53,10 @@
JITThunks();
~JITThunks() final;
+#if ENABLE(EXTRA_CTI_THUNKS)
+ void preinitializeExtraCTIThunks(VM&);
+#endif
+
MacroAssemblerCodePtr<JITThunkPtrTag> ctiNativeCall(VM&);
MacroAssemblerCodePtr<JITThunkPtrTag> ctiNativeConstruct(VM&);
MacroAssemblerCodePtr<JITThunkPtrTag> ctiNativeTailCall(VM&);
@@ -61,6 +66,10 @@
MacroAssemblerCodeRef<JITThunkPtrTag> ctiStub(VM&, ThunkGenerator);
MacroAssemblerCodeRef<JITThunkPtrTag> existingCTIStub(ThunkGenerator);
+ MacroAssemblerCodeRef<JITThunkPtrTag> existingCTIStub(ThunkGenerator, NoLockingNecessaryTag);
+#if ENABLE(EXTRA_CTI_THUNKS)
+ MacroAssemblerCodeRef<JITThunkPtrTag> ctiSlowPathFunctionStub(VM&, SlowPathFunction);
+#endif
NativeExecutable* hostFunctionStub(VM&, TaggedNativeFunction, TaggedNativeFunction constructor, const String& name);
NativeExecutable* hostFunctionStub(VM&, TaggedNativeFunction, TaggedNativeFunction constructor, ThunkGenerator, Intrinsic, const DOMJIT::Signature*, const String& name);
Added: trunk/Source/_javascript_Core/jit/SlowPathCall.cpp (0 => 277383)
--- trunk/Source/_javascript_Core/jit/SlowPathCall.cpp (rev 0)
+++ trunk/Source/_javascript_Core/jit/SlowPathCall.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -0,0 +1,98 @@
+/*
+ * Copyright (C) 2021 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "SlowPathCall.h"
+
+#if ENABLE(JIT)
+
+#include "CCallHelpers.h"
+#include "JITThunks.h"
+#include "VM.h"
+
+namespace JSC {
+
+#if ENABLE(EXTRA_CTI_THUNKS)
+
+void JITSlowPathCall::call()
+{
+ VM& vm = m_jit->vm();
+ uint32_t bytecodeOffset = m_jit->m_bytecodeIndex.offset();
+ ASSERT((bytecodeOffset << BytecodeIndex::checkpointShift) == m_jit->m_bytecodeIndex.asBits());
+
+ UNUSED_VARIABLE(m_pc);
+ constexpr GPRReg bytecodeOffsetReg = GPRInfo::argumentGPR1;
+ m_jit->move(JIT::TrustedImm32(bytecodeOffset), bytecodeOffsetReg);
+ m_jit->emitNakedNearCall(vm.jitStubs->ctiSlowPathFunctionStub(vm, m_slowPathFunction).retaggedCode<NoPtrTag>());
+}
+
+MacroAssemblerCodeRef<JITThunkPtrTag> JITSlowPathCall::generateThunk(VM& vm, SlowPathFunction slowPathFunction)
+{
+ CCallHelpers jit;
+
+ constexpr GPRReg bytecodeOffsetReg = JIT::argumentGPR1;
+
+#if CPU(X86_64)
+ jit.push(X86Registers::ebp);
+#elif CPU(ARM64)
+ jit.tagReturnAddress();
+ jit.pushPair(CCallHelpers::framePointerRegister, CCallHelpers::linkRegister);
+#endif
+
+ jit.store32(bytecodeOffsetReg, CCallHelpers::tagFor(CallFrameSlot::argumentCountIncludingThis));
+
+ jit.loadPtr(CCallHelpers::addressFor(CallFrameSlot::codeBlock), GPRInfo::argumentGPR0);
+
+ jit.prepareCallOperation(vm);
+
+ jit.loadPtr(CCallHelpers::Address(GPRInfo::argumentGPR0, CodeBlock::offsetOfInstructionsRawPointer()), GPRInfo::argumentGPR0);
+ static_assert(JIT::argumentGPR1 == bytecodeOffsetReg);
+ jit.addPtr(GPRInfo::argumentGPR0, GPRInfo::argumentGPR1);
+ jit.move(GPRInfo::callFrameRegister, GPRInfo::argumentGPR0);
+
+ CCallHelpers::Call call = jit.call(OperationPtrTag);
+
+ CCallHelpers::Jump exceptionCheck = jit.emitExceptionCheck(vm);
+
+#if CPU(X86_64)
+ jit.pop(X86Registers::ebp);
+#elif CPU(ARM64)
+ jit.popPair(CCallHelpers::framePointerRegister, CCallHelpers::linkRegister);
+#endif
+ jit.ret();
+
+ auto handler = vm.jitStubs->existingCTIStub(popThunkStackPreservesAndHandleExceptionGenerator, NoLockingNecessary);
+
+ LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::Thunk);
+ patchBuffer.link(call, FunctionPtr<OperationPtrTag>(slowPathFunction));
+ patchBuffer.link(exceptionCheck, CodeLocationLabel(handler.retaggedCode<NoPtrTag>()));
+ return FINALIZE_CODE(patchBuffer, JITThunkPtrTag, "SlowPathCall");
+}
+
+#endif // ENABLE(EXTRA_CTI_THUNKS)
+
+} // namespace JSC
+
+#endif // ENABLE(JIT)
Modified: trunk/Source/_javascript_Core/jit/SlowPathCall.h (277382 => 277383)
--- trunk/Source/_javascript_Core/jit/SlowPathCall.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/jit/SlowPathCall.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013-2018 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2021 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -25,8 +25,9 @@
#pragma once
-#include "CommonSlowPaths.h"
+#include "JIT.h"
#include "MacroAssemblerCodeRef.h"
+#include "SlowPathFunction.h"
#if ENABLE(JIT)
@@ -42,6 +43,12 @@
assertIsCFunctionPtr(slowPathFunction);
}
+#if ENABLE(EXTRA_CTI_THUNKS)
+ void call();
+ static MacroAssemblerCodeRef<JITThunkPtrTag> generateThunk(VM&, SlowPathFunction);
+
+#else // not ENABLE(EXTRA_CTI_THUNKS)
+
JIT::Call call()
{
m_jit->updateTopCallFrame();
@@ -62,10 +69,11 @@
static_assert(JIT::regT0 == GPRInfo::returnValueGPR);
static_assert(JIT::regT1 == GPRInfo::returnValueGPR2);
#endif
-
+
m_jit->exceptionCheck();
return call;
}
+#endif // ENABLE(EXTRA_CTI_THUNKS)
private:
JIT* m_jit;
Modified: trunk/Source/_javascript_Core/jit/ThunkGenerators.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/jit/ThunkGenerators.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/jit/ThunkGenerators.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -33,6 +33,7 @@
#include "MaxFrameExtentForSlowPathCall.h"
#include "SpecializedThunkJIT.h"
#include <wtf/InlineASM.h>
+#include <wtf/StdIntExtras.h>
#include <wtf/StringPrintStream.h>
#include <wtf/text/StringImpl.h>
@@ -40,6 +41,63 @@
namespace JSC {
+#if ENABLE(EXTRA_CTI_THUNKS)
+
+MacroAssemblerCodeRef<JITThunkPtrTag> handleExceptionGenerator(VM& vm)
+{
+ CCallHelpers jit;
+
+ jit.copyCalleeSavesToEntryFrameCalleeSavesBuffer(vm.topEntryFrame);
+
+ jit.move(CCallHelpers::TrustedImmPtr(&vm), GPRInfo::argumentGPR0);
+ jit.prepareCallOperation(vm);
+
+ CCallHelpers::Call operation = jit.call(OperationPtrTag);
+
+ jit.jumpToExceptionHandler(vm);
+
+ LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::Thunk);
+ patchBuffer.link(operation, FunctionPtr<OperationPtrTag>(operationLookupExceptionHandler));
+ return FINALIZE_CODE(patchBuffer, JITThunkPtrTag, "handleException");
+}
+
+MacroAssemblerCodeRef<JITThunkPtrTag> handleExceptionWithCallFrameRollbackGenerator(VM& vm)
+{
+ CCallHelpers jit;
+
+ jit.copyCalleeSavesToEntryFrameCalleeSavesBuffer(vm.topEntryFrame);
+
+ jit.move(CCallHelpers::TrustedImmPtr(&vm), GPRInfo::argumentGPR0);
+ jit.prepareCallOperation(vm);
+ CCallHelpers::Call operation = jit.call(OperationPtrTag);
+ jit.jumpToExceptionHandler(vm);
+
+ LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::Thunk);
+ patchBuffer.link(operation, FunctionPtr<OperationPtrTag>(operationLookupExceptionHandlerFromCallerFrame));
+ return FINALIZE_CODE(patchBuffer, JITThunkPtrTag, "handleExceptionWithCallFrameRollback");
+}
+
+MacroAssemblerCodeRef<JITThunkPtrTag> popThunkStackPreservesAndHandleExceptionGenerator(VM& vm)
+{
+ CCallHelpers jit;
+
+#if CPU(X86_64)
+ jit.addPtr(CCallHelpers::TrustedImm32(2 * sizeof(CPURegister)), X86Registers::esp);
+#elif CPU(ARM64)
+ jit.popPair(CCallHelpers::framePointerRegister, CCallHelpers::linkRegister);
+#endif
+
+ CCallHelpers::Jump continuation = jit.jump();
+
+ LinkBuffer patchBuffer(jit, GLOBAL_THUNK_ID, LinkBuffer::Profile::Thunk);
+ auto handler = vm.jitStubs->existingCTIStub(handleExceptionGenerator, NoLockingNecessary);
+ RELEASE_ASSERT(handler);
+ patchBuffer.link(continuation, CodeLocationLabel(handler.retaggedCode<NoPtrTag>()));
+ return FINALIZE_CODE(patchBuffer, JITThunkPtrTag, "popThunkStackPreservesAndHandleException");
+}
+
+#endif // ENABLE(EXTRA_CTI_THUNKS)
+
template<typename TagType>
inline void emitPointerValidation(CCallHelpers& jit, GPRReg pointerGPR, TagType tag)
{
Modified: trunk/Source/_javascript_Core/jit/ThunkGenerators.h (277382 => 277383)
--- trunk/Source/_javascript_Core/jit/ThunkGenerators.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/jit/ThunkGenerators.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2010-2019 Apple Inc. All rights reserved.
+ * Copyright (C) 2010-2021 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -35,6 +35,15 @@
template<PtrTag> class MacroAssemblerCodeRef;
class VM;
+#if ENABLE(EXTRA_CTI_THUNKS)
+
+MacroAssemblerCodeRef<JITThunkPtrTag> handleExceptionGenerator(VM&);
+MacroAssemblerCodeRef<JITThunkPtrTag> handleExceptionWithCallFrameRollbackGenerator(VM&);
+MacroAssemblerCodeRef<JITThunkPtrTag> popThunkStackPreservesAndHandleExceptionGenerator(VM&);
+
+#endif // ENABLE(EXTRA_CTI_THUNKS)
+
+
MacroAssemblerCodeRef<JITThunkPtrTag> throwExceptionFromCallSlowPathGenerator(VM&);
MacroAssemblerCodeRef<JITThunkPtrTag> linkCallThunkGenerator(VM&);
Modified: trunk/Source/_javascript_Core/runtime/CommonSlowPaths.h (277382 => 277383)
--- trunk/Source/_javascript_Core/runtime/CommonSlowPaths.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/runtime/CommonSlowPaths.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2011-2019 Apple Inc. All rights reserved.
+ * Copyright (C) 2011-2021 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -32,7 +32,7 @@
#include "FunctionCodeBlock.h"
#include "JSImmutableButterfly.h"
#include "ScopedArguments.h"
-#include "SlowPathReturnType.h"
+#include "SlowPathFunction.h"
#include "StackAlignment.h"
#include "VMInlines.h"
#include <wtf/StdLibExtras.h>
@@ -299,6 +299,4 @@
JSC_DECLARE_COMMON_SLOW_PATH(iterator_next_try_fast_wide16);
JSC_DECLARE_COMMON_SLOW_PATH(iterator_next_try_fast_wide32);
-using SlowPathFunction = SlowPathReturnType(JIT_OPERATION_ATTRIBUTES*)(CallFrame*, const Instruction*);
-
} // namespace JSC
Added: trunk/Source/_javascript_Core/runtime/SlowPathFunction.h (0 => 277383)
--- trunk/Source/_javascript_Core/runtime/SlowPathFunction.h (rev 0)
+++ trunk/Source/_javascript_Core/runtime/SlowPathFunction.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -0,0 +1,37 @@
+/*
+ * Copyright (C) 2021 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include "SlowPathReturnType.h"
+
+namespace JSC {
+
+class CallFrame;
+struct Instruction;
+
+using SlowPathFunction = SlowPathReturnType(JIT_OPERATION_ATTRIBUTES*)(CallFrame*, const Instruction*);
+
+} // namespace JSC
Modified: trunk/Source/_javascript_Core/runtime/VM.cpp (277382 => 277383)
--- trunk/Source/_javascript_Core/runtime/VM.cpp 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/_javascript_Core/runtime/VM.cpp 2021-05-12 19:11:25 UTC (rev 277383)
@@ -567,8 +567,12 @@
#endif // ENABLE(FTL_JIT)
getCTIInternalFunctionTrampolineFor(CodeForCall);
getCTIInternalFunctionTrampolineFor(CodeForConstruct);
+
+#if ENABLE(EXTRA_CTI_THUNKS)
+ jitStubs->preinitializeExtraCTIThunks(*this);
+#endif
}
-#endif
+#endif // ENABLE(JIT)
if (Options::forceDebuggerBytecodeGeneration() || Options::alwaysUseShadowChicken())
ensureShadowChicken();
Modified: trunk/Source/WTF/ChangeLog (277382 => 277383)
--- trunk/Source/WTF/ChangeLog 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/WTF/ChangeLog 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,3 +1,19 @@
+2021-05-12 Mark Lam <[email protected]>
+
+ Implement some common Baseline JIT slow paths using JIT thunks.
+ https://bugs.webkit.org/show_bug.cgi?id=225682
+
+ Reviewed by Filip Pizlo.
+
+ Introduce ENABLE(EXTRA_CTI_THUNKS) flag to guard the use of these new thunks.
+ Currently, the thunks are 64-bit only, and only supported for ARM64 and non-Windows
+ X86_64. The reason it is not supported for Windows as well is because Windows
+ only has 4 argument registers. In this patch, the thunks do not use that many
+ registers yet, but there will be more thunks coming that will require the use
+ of up to 6 argument registers.
+
+ * wtf/PlatformEnable.h:
+
2021-05-11 Chris Dumez <[email protected]>
Port WTF::FileSystem::listDirectory to std::filesystem
Modified: trunk/Source/WTF/wtf/PlatformEnable.h (277382 => 277383)
--- trunk/Source/WTF/wtf/PlatformEnable.h 2021-05-12 18:40:38 UTC (rev 277382)
+++ trunk/Source/WTF/wtf/PlatformEnable.h 2021-05-12 19:11:25 UTC (rev 277383)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2006-2020 Apple Inc. All rights reserved.
+ * Copyright (C) 2006-2021 Apple Inc. All rights reserved.
* Copyright (C) 2007-2009 Torch Mobile, Inc.
* Copyright (C) 2010, 2011 Research In Motion Limited. All rights reserved.
* Copyright (C) 2013 Samsung Electronics. All rights reserved.
@@ -832,6 +832,14 @@
#define ENABLE_JIT_OPERATION_VALIDATION 1
#endif
+#if CPU(ARM64) || (CPU(X86_64) && !OS(WINDOWS))
+/* The implementation of these thunks can use up to 6 argument registers, and
+ make use of ARM64 like features. For now, we'll only support them on platforms
+ that have 6 or more argument registers to use.
+*/
+#define ENABLE_EXTRA_CTI_THUNKS 1
+#endif
+
#if !defined(ENABLE_BINDING_INTEGRITY) && !OS(WINDOWS)
#define ENABLE_BINDING_INTEGRITY 1
#endif
