- Revision
- 190569
- Author
- [email protected]
- Date
- 2015-10-05 12:35:32 -0700 (Mon, 05 Oct 2015)
Log Message
Allow an object's marking state to track The Three Colors
https://bugs.webkit.org/show_bug.cgi?id=149654
Reviewed by Geoffrey Garen.
Source/_javascript_Core:
I want to make GC marking concurrent (see https://bugs.webkit.org/show_bug.cgi?id=149432).
Concurrent GC require barriers to be executed during certain heap operations. We already have a
generational GC. Generational GCs also need barriers, and we already have those. The generational
GC barrier that we use is the "sticky mark bit" barrier. Ordinarily, mark bits get reset after a
collection. In our collector, there is a secondary mark bit that "sticks" - i.e. it does not get
reset. If the sticky mark bit is set in between two collections, then we know that the object is in
old space. This is sufficient to determine when to put things into remembered sets. Additionally,
the sticky mark bit is actually a tri-state that can also tell us if the object has been placed on
a remembered set.
This is awfully similar to what you want in a concurrent GC. Concurrent GCs typically want writes
to the heap that change the object graph to do different things depending on an object's marking
state, which is usually referred to as its color. White means that the object has never been seen
by the collector. All white objects are presumed dead at the flip. Grey objects are those that are
known to the collector but have not been scanned. Black objects are those that have been scanned,
and will not be scanned again. White is exactly just "not being marked", and both grey and black
mean "marked" - with "black" meaning "marked but not on any worklist". That's quite a bit like the
current "Marked" and "MarkedAndRemembered" states that we have for generational GC.
"MarkedAndRemembered" is a lot like "grey", and "Marked" is a lot like "black".
I want to make a concurrent GC that unifies the generational and concurrent barriers into a single
fast path check. Even better if the two barriers are entirely identical. You can do this using
Pirinen's technique #2 [1], originally due to Guy Steele [2]: when doing o.f=v where o is black and
v is white, turn o grey again. This is like remembering an object, in the sense that our gen GC
"rememberes" o when o is old and v is new. It remembers objects by putting them on the mark stack,
setting the generational state to MarkedAndRemembered, and doing nothing to the primary mark bit.
This makes our concurrent GC approach pretty obvious. We want to use one barrier for concurrent and
generational, and we want to basically keep our current barriers unchanged. The only things missing
are just some small changes to allow the concurrent GC to know precisely when an object is black,
and to know during object visiting if we are visiting the object for the first time during a
collection or a subsequent time due to barrier re-greying (concurrent GC) or barrier remembering
(generational GC). So, this patch does the following:
- Changes the terminology used for the gcData header byte in JSCell. This changes the name of this
to cellState, and introduces a new enumeration called CellState. This new enumeration behaves a
lot like the old GCData did. It has the following members, with the following correspondence to
the old GCData:
OldBlack: this is like Marked, with the exception that we ensure that an object becomes OldBlack
as soon as the object starts to be scanned. Previously, an object might be
MarkedAndRemembered during scanning and we'd turn all MarkedAndRemembered objects into Marked
objects during a post-processing step at the end of GC. This patch gets rid of that
post-processing. The act of visiting an object unconditionally makes it OldBlack. Note that
our definition of "black" is not that the object is done being scanned, but that it is either
being scanned right now or it has already been scanned. This is like a combination of
Siebert's anthracite and black states [3].
NewWhite: this is exactly NotMarked. It's the state that objects get when they are allocated.
It's impossible for an object to return to this state.
OldGrey: the object is on the mark stack and will be scanned at some point in the future. This
also means that this isn't the first time in this cycle that the object has been grey. In an
eden collection, an old object that has been remembered is thought of as being OldGrey, even
if this is the first time during this eden collection that it is grey. That's because an eden
collection must behave "as if" the grey->black transition for old objects magically happened
at the start of GC. Remembered objects are like old objects that underwent a concurrent
barrier re-greying just after the magical old object grey->black transition at the start of
GC. This state is almost exactly like MarkedAndRemembered, except that an object now
transitions from OldGrey to OldBlack at the beginning of visiting, rather than how previously
we transitioned from MarkedAndRemembered to Marked at the bitter end of GC.
NewGray: the object is on the mark stack and will be scanned at some point in the future. This
state has no clear relative in the old state system. It means that the object became grey due
to ordinary marking. Previously, ordinary marking would make the object Marked.
- Removal of the post-processing phase that "clears" the remembered set by moving all remembered
objects to the Marked state. This now happens magically during visiting, as described above.
- SlotVisitor now remembers the state that the object did have just before visiting. While visiting
that object, it's possible to query what the state was. This is used for copy space decisions and
for extra memory usage accounting. We don't want to put the backing store on the copy worklist,
and we don't want to count extra memory usage, if the object was OldGrey at the start of
visiting. Previously, we would be able to just ask if the object was MarkedAndRemembered since
that state wouldn't get cleared until after all marking finished. This change also simplifies
some APIs, because there is no need to pass the JSCell* pointer, since these SlotVisitor methods
no longer ask the cell for its state - instead they use the saved pre-visiting state.
- Removal of a bunch of helpers and abstractions. Previously we had various methods for asking if
an object was "marked" and if an object was "remembered". We had helpers for adjusting these
states, and those helpers would assert that they were being used the right way. This is not very
useful for concurrent GC, since now the set of possible state transitions is much larger. Also,
the previous use of the word "marked" was pretty bad - for example in Heap, "marked" refers to
the primary mark bit (that gets cleared at the flip), while in JSCell, "marked" refers to the
sticky mark bit (that does not get cleared, ever). This change gets rid of a lot of those helpers
and inlines their logic. This actually makes the code easier and more fun to read, since you can
now look at the marking and barrier code and see how that code uses the four CellStates. For
example, it's fun to see that the barrier gets fired for o.f=v exactly when o is OldBlack and v
is NewWhite.
This change shouldn't have any effect on performance or GC behavior. It does put our code in a
weird state where we now have states and comments referencing a concurrent GC that doesn't exist
yet.
Finally, some thoughts about the concurrent GC barrier and its implications for performance. This
barrier exhibits very poor guarantees about collector progress, but maximizes throughput by just
reusing the existing barrier code we already emit and optimize. I believe that even our epoch-based
barrier insertion DFG phase is correct for the concurrent interpretation of our existing barrier.
But, the barrier can regress the progress that the collector has made for two reasons:
Incremental update: you don't want to use this barrier with a black stack, since that would mean
that heap loads of white objects will have to explicitly re-grey the stack. The way you implement
this kind of collector is that collector termination will rescan the stack. Termination is reached
only if the at-termination re-scan greys no objects. This means that the collector is a fixpoint.
Luckily, our collector is already a fixpoint because of opaque roots and structure transitions.
Marking ain't monotonic: normally, once an object is black, it stays that way. In this collector,
black objects may become grey again. I don't have personal experience with such concurrent GCs, but
I suspect that this will basically be fine. Concurrent collections finish pretty quickly, and the
mutator usually touches only a subset of the heap. Only that subset of the heap that the mutator is
touching could be re-greyed. Probably, the GC will have to be hybrid incremental and concurrent,
and towards the end of GC when we do the termination stack re-scan, we can ensure that the
collector does some minimal amount of marking. If the minimal amount of marking done by the
collector is large enough, we can ensure that we reach termination before the mutator can regress
progress. The barrier cannot un-terminate the collector; if the collector reaches termination and
the barrier re-greys an object then it's actually doing a generational remembering rather than a
concurrent re-greying.
That's sort of the cute thing about the barrier - it is exactly a re-greying barrier during GC and
it is exactly a remembering barrier in between GCs.
[1] http://www.cs.utexas.edu/ftp/garbage/submit/readable/ppirinen11.ps
[2] http://dl.acm.org/citation.cfm?id=361005
[3] http://www.aicas.com/papers/ISMM132-siebert.pdf
* _javascript_Core.vcxproj/_javascript_Core.vcxproj:
* _javascript_Core.vcxproj/_javascript_Core.vcxproj.filters:
* _javascript_Core.xcodeproj/project.pbxproj:
* bytecode/CodeBlock.cpp:
(JSC::CodeBlock::visitChildren):
* ftl/FTLAbstractHeapRepository.cpp:
(JSC::FTL::AbstractHeapRepository::AbstractHeapRepository):
* ftl/FTLAbstractHeapRepository.h:
* ftl/FTLLowerDFGToLLVM.cpp:
(JSC::FTL::DFG::LowerDFGToLLVM::masqueradesAsUndefinedWatchpointIsStillValid):
(JSC::FTL::DFG::LowerDFGToLLVM::loadCellState):
(JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier):
(JSC::FTL::DFG::LowerDFGToLLVM::loadMarkByte): Deleted.
* heap/CellState.h: Added.
* heap/CodeBlockSet.cpp:
(JSC::CodeBlockSet::rememberCurrentlyExecutingCodeBlocks):
* heap/CopiedBlock.h:
* heap/CopiedBlockInlines.h:
(JSC::CopiedBlock::reportLiveBytes):
(JSC::CopiedBlock::shouldReportLiveBytes): Deleted.
* heap/GCLogging.cpp:
(JSC::LoggingFunctor::reviveCells):
* heap/Heap.cpp:
(JSC::Heap::markRoots):
(JSC::Heap::visitWeakHandles):
(JSC::Heap::updateObjectCounts):
(JSC::Heap::addToRememberedSet):
(JSC::Heap::clearRememberedSet): Deleted.
* heap/Heap.h:
* heap/HeapInlines.h:
(JSC::Heap::isLive):
(JSC::Heap::isMarked):
(JSC::Heap::writeBarrier):
(JSC::Heap::reportExtraMemoryAllocated):
(JSC::Heap::reportExtraMemoryVisited):
(JSC::Heap::isRemembered): Deleted.
* heap/SlotVisitor.cpp:
(JSC::SlotVisitor::append):
(JSC::SlotVisitor::visitChildren):
(JSC::SlotVisitor::donateKnownParallel):
(JSC::SlotVisitor::drain):
(JSC::visitChildren): Deleted.
* heap/SlotVisitor.h:
(JSC::SlotVisitor::childCount):
(JSC::SlotVisitor::incrementChildCount):
(JSC::SlotVisitor::dataBeforeVisitingCurrentObject):
* heap/SlotVisitorInlines.h:
(JSC::SlotVisitor::internalAppend):
(JSC::SlotVisitor::copyLater):
(JSC::SlotVisitor::reportExtraMemoryVisited):
(JSC::SlotVisitor::heap):
* jit/AssemblyHelpers.h:
(JSC::AssemblyHelpers::jumpIfIsRememberedOrInEden):
* llint/LowLevelInterpreter.asm:
* llint/LowLevelInterpreter32_64.asm:
* llint/LowLevelInterpreter64.asm:
* runtime/JSCell.h:
(JSC::JSCell::cellState):
(JSC::JSCell::setCellState):
(JSC::JSCell::structureIDOffset):
(JSC::JSCell::indexingTypeOffset):
(JSC::JSCell::cellStateOffset):
(JSC::JSCell::setMarked): Deleted.
(JSC::JSCell::setRemembered): Deleted.
(JSC::JSCell::isMarked): Deleted.
(JSC::JSCell::isRemembered): Deleted.
(JSC::JSCell::gcDataOffset): Deleted.
* runtime/JSCellInlines.h:
(JSC::JSCell::JSCell):
* runtime/JSGenericTypedArrayViewInlines.h:
(JSC::JSGenericTypedArrayView<Adaptor>::visitChildren):
* runtime/JSObject.cpp:
(JSC::JSObject::copyBackingStore):
* runtime/JSString.cpp:
(JSC::JSString::visitChildren):
* runtime/StructureIDBlob.h:
(JSC::StructureIDBlob::StructureIDBlob):
(JSC::StructureIDBlob::operator=):
* runtime/WeakMapData.cpp:
(JSC::WeakMapData::visitChildren):
(JSC::WeakMapData::set):
* tests/stress/basic-eden-gc-test.js: Added.
Hilariously, an earlier version of this patch that didn't have the NewGrey/OldGrey distinction
would only crash super-big tests that GCd twice but it didn't crash any small focused test. All
it took to show the need for the NewGrey/OldGrey distinction was this super simple test.
Source/WebCore:
No new tests because no new behavior.
* bindings/scripts/CodeGeneratorJS.pm:
(GenerateImplementation):
Modified Paths
Added Paths
Diff
Modified: trunk/Source/_javascript_Core/ChangeLog (190568 => 190569)
--- trunk/Source/_javascript_Core/ChangeLog 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/ChangeLog 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,3 +1,221 @@
+2015-10-03 Filip Pizlo <[email protected]>
+
+ Allow an object's marking state to track The Three Colors
+ https://bugs.webkit.org/show_bug.cgi?id=149654
+
+ Reviewed by Geoffrey Garen.
+
+ I want to make GC marking concurrent (see https://bugs.webkit.org/show_bug.cgi?id=149432).
+ Concurrent GC require barriers to be executed during certain heap operations. We already have a
+ generational GC. Generational GCs also need barriers, and we already have those. The generational
+ GC barrier that we use is the "sticky mark bit" barrier. Ordinarily, mark bits get reset after a
+ collection. In our collector, there is a secondary mark bit that "sticks" - i.e. it does not get
+ reset. If the sticky mark bit is set in between two collections, then we know that the object is in
+ old space. This is sufficient to determine when to put things into remembered sets. Additionally,
+ the sticky mark bit is actually a tri-state that can also tell us if the object has been placed on
+ a remembered set.
+
+ This is awfully similar to what you want in a concurrent GC. Concurrent GCs typically want writes
+ to the heap that change the object graph to do different things depending on an object's marking
+ state, which is usually referred to as its color. White means that the object has never been seen
+ by the collector. All white objects are presumed dead at the flip. Grey objects are those that are
+ known to the collector but have not been scanned. Black objects are those that have been scanned,
+ and will not be scanned again. White is exactly just "not being marked", and both grey and black
+ mean "marked" - with "black" meaning "marked but not on any worklist". That's quite a bit like the
+ current "Marked" and "MarkedAndRemembered" states that we have for generational GC.
+ "MarkedAndRemembered" is a lot like "grey", and "Marked" is a lot like "black".
+
+ I want to make a concurrent GC that unifies the generational and concurrent barriers into a single
+ fast path check. Even better if the two barriers are entirely identical. You can do this using
+ Pirinen's technique #2 [1], originally due to Guy Steele [2]: when doing o.f=v where o is black and
+ v is white, turn o grey again. This is like remembering an object, in the sense that our gen GC
+ "rememberes" o when o is old and v is new. It remembers objects by putting them on the mark stack,
+ setting the generational state to MarkedAndRemembered, and doing nothing to the primary mark bit.
+
+ This makes our concurrent GC approach pretty obvious. We want to use one barrier for concurrent and
+ generational, and we want to basically keep our current barriers unchanged. The only things missing
+ are just some small changes to allow the concurrent GC to know precisely when an object is black,
+ and to know during object visiting if we are visiting the object for the first time during a
+ collection or a subsequent time due to barrier re-greying (concurrent GC) or barrier remembering
+ (generational GC). So, this patch does the following:
+
+ - Changes the terminology used for the gcData header byte in JSCell. This changes the name of this
+ to cellState, and introduces a new enumeration called CellState. This new enumeration behaves a
+ lot like the old GCData did. It has the following members, with the following correspondence to
+ the old GCData:
+
+ OldBlack: this is like Marked, with the exception that we ensure that an object becomes OldBlack
+ as soon as the object starts to be scanned. Previously, an object might be
+ MarkedAndRemembered during scanning and we'd turn all MarkedAndRemembered objects into Marked
+ objects during a post-processing step at the end of GC. This patch gets rid of that
+ post-processing. The act of visiting an object unconditionally makes it OldBlack. Note that
+ our definition of "black" is not that the object is done being scanned, but that it is either
+ being scanned right now or it has already been scanned. This is like a combination of
+ Siebert's anthracite and black states [3].
+
+ NewWhite: this is exactly NotMarked. It's the state that objects get when they are allocated.
+ It's impossible for an object to return to this state.
+
+ OldGrey: the object is on the mark stack and will be scanned at some point in the future. This
+ also means that this isn't the first time in this cycle that the object has been grey. In an
+ eden collection, an old object that has been remembered is thought of as being OldGrey, even
+ if this is the first time during this eden collection that it is grey. That's because an eden
+ collection must behave "as if" the grey->black transition for old objects magically happened
+ at the start of GC. Remembered objects are like old objects that underwent a concurrent
+ barrier re-greying just after the magical old object grey->black transition at the start of
+ GC. This state is almost exactly like MarkedAndRemembered, except that an object now
+ transitions from OldGrey to OldBlack at the beginning of visiting, rather than how previously
+ we transitioned from MarkedAndRemembered to Marked at the bitter end of GC.
+
+ NewGray: the object is on the mark stack and will be scanned at some point in the future. This
+ state has no clear relative in the old state system. It means that the object became grey due
+ to ordinary marking. Previously, ordinary marking would make the object Marked.
+
+ - Removal of the post-processing phase that "clears" the remembered set by moving all remembered
+ objects to the Marked state. This now happens magically during visiting, as described above.
+
+ - SlotVisitor now remembers the state that the object did have just before visiting. While visiting
+ that object, it's possible to query what the state was. This is used for copy space decisions and
+ for extra memory usage accounting. We don't want to put the backing store on the copy worklist,
+ and we don't want to count extra memory usage, if the object was OldGrey at the start of
+ visiting. Previously, we would be able to just ask if the object was MarkedAndRemembered since
+ that state wouldn't get cleared until after all marking finished. This change also simplifies
+ some APIs, because there is no need to pass the JSCell* pointer, since these SlotVisitor methods
+ no longer ask the cell for its state - instead they use the saved pre-visiting state.
+
+ - Removal of a bunch of helpers and abstractions. Previously we had various methods for asking if
+ an object was "marked" and if an object was "remembered". We had helpers for adjusting these
+ states, and those helpers would assert that they were being used the right way. This is not very
+ useful for concurrent GC, since now the set of possible state transitions is much larger. Also,
+ the previous use of the word "marked" was pretty bad - for example in Heap, "marked" refers to
+ the primary mark bit (that gets cleared at the flip), while in JSCell, "marked" refers to the
+ sticky mark bit (that does not get cleared, ever). This change gets rid of a lot of those helpers
+ and inlines their logic. This actually makes the code easier and more fun to read, since you can
+ now look at the marking and barrier code and see how that code uses the four CellStates. For
+ example, it's fun to see that the barrier gets fired for o.f=v exactly when o is OldBlack and v
+ is NewWhite.
+
+ This change shouldn't have any effect on performance or GC behavior. It does put our code in a
+ weird state where we now have states and comments referencing a concurrent GC that doesn't exist
+ yet.
+
+ Finally, some thoughts about the concurrent GC barrier and its implications for performance. This
+ barrier exhibits very poor guarantees about collector progress, but maximizes throughput by just
+ reusing the existing barrier code we already emit and optimize. I believe that even our epoch-based
+ barrier insertion DFG phase is correct for the concurrent interpretation of our existing barrier.
+ But, the barrier can regress the progress that the collector has made for two reasons:
+
+ Incremental update: you don't want to use this barrier with a black stack, since that would mean
+ that heap loads of white objects will have to explicitly re-grey the stack. The way you implement
+ this kind of collector is that collector termination will rescan the stack. Termination is reached
+ only if the at-termination re-scan greys no objects. This means that the collector is a fixpoint.
+ Luckily, our collector is already a fixpoint because of opaque roots and structure transitions.
+
+ Marking ain't monotonic: normally, once an object is black, it stays that way. In this collector,
+ black objects may become grey again. I don't have personal experience with such concurrent GCs, but
+ I suspect that this will basically be fine. Concurrent collections finish pretty quickly, and the
+ mutator usually touches only a subset of the heap. Only that subset of the heap that the mutator is
+ touching could be re-greyed. Probably, the GC will have to be hybrid incremental and concurrent,
+ and towards the end of GC when we do the termination stack re-scan, we can ensure that the
+ collector does some minimal amount of marking. If the minimal amount of marking done by the
+ collector is large enough, we can ensure that we reach termination before the mutator can regress
+ progress. The barrier cannot un-terminate the collector; if the collector reaches termination and
+ the barrier re-greys an object then it's actually doing a generational remembering rather than a
+ concurrent re-greying.
+
+ That's sort of the cute thing about the barrier - it is exactly a re-greying barrier during GC and
+ it is exactly a remembering barrier in between GCs.
+
+ [1] http://www.cs.utexas.edu/ftp/garbage/submit/readable/ppirinen11.ps
+ [2] http://dl.acm.org/citation.cfm?id=361005
+ [3] http://www.aicas.com/papers/ISMM132-siebert.pdf
+
+ * _javascript_Core.vcxproj/_javascript_Core.vcxproj:
+ * _javascript_Core.vcxproj/_javascript_Core.vcxproj.filters:
+ * _javascript_Core.xcodeproj/project.pbxproj:
+ * bytecode/CodeBlock.cpp:
+ (JSC::CodeBlock::visitChildren):
+ * ftl/FTLAbstractHeapRepository.cpp:
+ (JSC::FTL::AbstractHeapRepository::AbstractHeapRepository):
+ * ftl/FTLAbstractHeapRepository.h:
+ * ftl/FTLLowerDFGToLLVM.cpp:
+ (JSC::FTL::DFG::LowerDFGToLLVM::masqueradesAsUndefinedWatchpointIsStillValid):
+ (JSC::FTL::DFG::LowerDFGToLLVM::loadCellState):
+ (JSC::FTL::DFG::LowerDFGToLLVM::emitStoreBarrier):
+ (JSC::FTL::DFG::LowerDFGToLLVM::loadMarkByte): Deleted.
+ * heap/CellState.h: Added.
+ * heap/CodeBlockSet.cpp:
+ (JSC::CodeBlockSet::rememberCurrentlyExecutingCodeBlocks):
+ * heap/CopiedBlock.h:
+ * heap/CopiedBlockInlines.h:
+ (JSC::CopiedBlock::reportLiveBytes):
+ (JSC::CopiedBlock::shouldReportLiveBytes): Deleted.
+ * heap/GCLogging.cpp:
+ (JSC::LoggingFunctor::reviveCells):
+ * heap/Heap.cpp:
+ (JSC::Heap::markRoots):
+ (JSC::Heap::visitWeakHandles):
+ (JSC::Heap::updateObjectCounts):
+ (JSC::Heap::addToRememberedSet):
+ (JSC::Heap::clearRememberedSet): Deleted.
+ * heap/Heap.h:
+ * heap/HeapInlines.h:
+ (JSC::Heap::isLive):
+ (JSC::Heap::isMarked):
+ (JSC::Heap::writeBarrier):
+ (JSC::Heap::reportExtraMemoryAllocated):
+ (JSC::Heap::reportExtraMemoryVisited):
+ (JSC::Heap::isRemembered): Deleted.
+ * heap/SlotVisitor.cpp:
+ (JSC::SlotVisitor::append):
+ (JSC::SlotVisitor::visitChildren):
+ (JSC::SlotVisitor::donateKnownParallel):
+ (JSC::SlotVisitor::drain):
+ (JSC::visitChildren): Deleted.
+ * heap/SlotVisitor.h:
+ (JSC::SlotVisitor::childCount):
+ (JSC::SlotVisitor::incrementChildCount):
+ (JSC::SlotVisitor::dataBeforeVisitingCurrentObject):
+ * heap/SlotVisitorInlines.h:
+ (JSC::SlotVisitor::internalAppend):
+ (JSC::SlotVisitor::copyLater):
+ (JSC::SlotVisitor::reportExtraMemoryVisited):
+ (JSC::SlotVisitor::heap):
+ * jit/AssemblyHelpers.h:
+ (JSC::AssemblyHelpers::jumpIfIsRememberedOrInEden):
+ * llint/LowLevelInterpreter.asm:
+ * llint/LowLevelInterpreter32_64.asm:
+ * llint/LowLevelInterpreter64.asm:
+ * runtime/JSCell.h:
+ (JSC::JSCell::cellState):
+ (JSC::JSCell::setCellState):
+ (JSC::JSCell::structureIDOffset):
+ (JSC::JSCell::indexingTypeOffset):
+ (JSC::JSCell::cellStateOffset):
+ (JSC::JSCell::setMarked): Deleted.
+ (JSC::JSCell::setRemembered): Deleted.
+ (JSC::JSCell::isMarked): Deleted.
+ (JSC::JSCell::isRemembered): Deleted.
+ (JSC::JSCell::gcDataOffset): Deleted.
+ * runtime/JSCellInlines.h:
+ (JSC::JSCell::JSCell):
+ * runtime/JSGenericTypedArrayViewInlines.h:
+ (JSC::JSGenericTypedArrayView<Adaptor>::visitChildren):
+ * runtime/JSObject.cpp:
+ (JSC::JSObject::copyBackingStore):
+ * runtime/JSString.cpp:
+ (JSC::JSString::visitChildren):
+ * runtime/StructureIDBlob.h:
+ (JSC::StructureIDBlob::StructureIDBlob):
+ (JSC::StructureIDBlob::operator=):
+ * runtime/WeakMapData.cpp:
+ (JSC::WeakMapData::visitChildren):
+ (JSC::WeakMapData::set):
+ * tests/stress/basic-eden-gc-test.js: Added.
+ Hilariously, an earlier version of this patch that didn't have the NewGrey/OldGrey distinction
+ would only crash super-big tests that GCd twice but it didn't crash any small focused test. All
+ it took to show the need for the NewGrey/OldGrey distinction was this super simple test.
+
2015-10-05 Geoffrey Garen <[email protected]>
JSC::SlotVisitor should not be a hot mess
Modified: trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj (190568 => 190569)
--- trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1326,6 +1326,7 @@
<ClInclude Include="..\ftl\FTLValueRange.h" />
<ClInclude Include="..\ftl\FTLWeight.h" />
<ClInclude Include="..\ftl\FTLWeightedTarget.h" />
+ <ClInclude Include="..\heap\CellState.h" />
<ClInclude Include="..\heap\CodeBlockSet.h" />
<ClInclude Include="..\heap\ConservativeRoots.h" />
<ClInclude Include="..\heap\CopiedAllocator.h" />
Modified: trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj.filters (190568 => 190569)
--- trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj.filters 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/_javascript_Core.vcxproj/_javascript_Core.vcxproj.filters 2015-10-05 19:35:32 UTC (rev 190569)
@@ -2219,6 +2219,9 @@
<ClInclude Include="..\disassembler\Disassembler.h">
<Filter>disassembler</Filter>
</ClInclude>
+ <ClInclude Include="..\heap\CellState.h">
+ <Filter>heap</Filter>
+ </ClInclude>
<ClInclude Include="..\heap\ConservativeRoots.h">
<Filter>heap</Filter>
</ClInclude>
Modified: trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj (190568 => 190569)
--- trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/_javascript_Core.xcodeproj/project.pbxproj 2015-10-05 19:35:32 UTC (rev 190569)
@@ -115,6 +115,7 @@
0F18D3D01B55A6E0002C5C9F /* DFGAdaptiveStructureWatchpoint.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F18D3CE1B55A6E0002C5C9F /* DFGAdaptiveStructureWatchpoint.h */; };
0F190CAC189D82F6000AE5F0 /* ProfilerJettisonReason.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0F190CAA189D82F6000AE5F0 /* ProfilerJettisonReason.cpp */; };
0F190CAD189D82F6000AE5F0 /* ProfilerJettisonReason.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F190CAB189D82F6000AE5F0 /* ProfilerJettisonReason.h */; settings = {ATTRIBUTES = (Private, ); }; };
+ 0F1C3DDA1BBCE09E00E523E4 /* CellState.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F1C3DD91BBCE09E00E523E4 /* CellState.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F1DD84A18A945BE0026F3FA /* JSCInlines.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F1DD84918A945BE0026F3FA /* JSCInlines.h */; settings = {ATTRIBUTES = (Private, ); }; };
0F1E3A461534CBAF000F9456 /* DFGArgumentPosition.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F1E3A431534CBAD000F9456 /* DFGArgumentPosition.h */; };
0F1E3A471534CBB9000F9456 /* DFGDoubleFormatState.h in Headers */ = {isa = PBXBuildFile; fileRef = 0F1E3A441534CBAD000F9456 /* DFGDoubleFormatState.h */; };
@@ -1964,6 +1965,7 @@
0F18D3CE1B55A6E0002C5C9F /* DFGAdaptiveStructureWatchpoint.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGAdaptiveStructureWatchpoint.h; path = dfg/DFGAdaptiveStructureWatchpoint.h; sourceTree = "<group>"; };
0F190CAA189D82F6000AE5F0 /* ProfilerJettisonReason.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ProfilerJettisonReason.cpp; path = profiler/ProfilerJettisonReason.cpp; sourceTree = "<group>"; };
0F190CAB189D82F6000AE5F0 /* ProfilerJettisonReason.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = ProfilerJettisonReason.h; path = profiler/ProfilerJettisonReason.h; sourceTree = "<group>"; };
+ 0F1C3DD91BBCE09E00E523E4 /* CellState.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CellState.h; sourceTree = "<group>"; };
0F1DD84918A945BE0026F3FA /* JSCInlines.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSCInlines.h; sourceTree = "<group>"; };
0F1E3A431534CBAD000F9456 /* DFGArgumentPosition.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGArgumentPosition.h; path = dfg/DFGArgumentPosition.h; sourceTree = "<group>"; };
0F1E3A441534CBAD000F9456 /* DFGDoubleFormatState.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = DFGDoubleFormatState.h; path = dfg/DFGDoubleFormatState.h; sourceTree = "<group>"; };
@@ -4226,6 +4228,7 @@
142E312A134FF0A600AFADB5 /* heap */ = {
isa = PBXGroup;
children = (
+ 0F1C3DD91BBCE09E00E523E4 /* CellState.h */,
0FD8A31117D4326C00CA2C40 /* CodeBlockSet.cpp */,
0FD8A31217D4326C00CA2C40 /* CodeBlockSet.h */,
146B14DB12EB5B12001BEC1B /* ConservativeRoots.cpp */,
@@ -6917,6 +6920,7 @@
14142E511B796ECE00F4BF4B /* UnlinkedFunctionExecutable.h in Headers */,
0F2E892C16D028AD009E4FD2 /* UnusedPointer.h in Headers */,
0F963B3813FC6FE90002D9B2 /* ValueProfile.h in Headers */,
+ 0F1C3DDA1BBCE09E00E523E4 /* CellState.h in Headers */,
0F426A481460CBB300131F8F /* ValueRecovery.h in Headers */,
79EE0C001B4AFB85000385C9 /* VariableEnvironment.h in Headers */,
0F6C73511AC9F99F00BE1682 /* VariableWriteFireDetail.h in Headers */,
Modified: trunk/Source/_javascript_Core/bytecode/CodeBlock.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/bytecode/CodeBlock.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/bytecode/CodeBlock.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -2245,15 +2245,15 @@
if (CodeBlock* otherBlock = specialOSREntryBlockOrNull())
otherBlock->visitAggregate(visitor);
- visitor.reportExtraMemoryVisited(ownerExecutable(), sizeof(CodeBlock));
+ visitor.reportExtraMemoryVisited(sizeof(CodeBlock));
if (m_jitCode)
- visitor.reportExtraMemoryVisited(ownerExecutable(), m_jitCode->size());
+ visitor.reportExtraMemoryVisited(m_jitCode->size());
if (m_instructions.size()) {
// Divide by refCount() because m_instructions points to something that is shared
// by multiple CodeBlocks, and we only want to count it towards the heap size once.
// Having each CodeBlock report only its proportional share of the size is one way
// of accomplishing this.
- visitor.reportExtraMemoryVisited(ownerExecutable(), m_instructions.size() * sizeof(Instruction) / m_instructions.refCount());
+ visitor.reportExtraMemoryVisited(m_instructions.size() * sizeof(Instruction) / m_instructions.refCount());
}
visitor.append(&m_unlinkedCode);
Modified: trunk/Source/_javascript_Core/ftl/FTLAbstractHeapRepository.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/ftl/FTLAbstractHeapRepository.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/ftl/FTLAbstractHeapRepository.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -69,12 +69,12 @@
RELEASE_ASSERT(!(JSCell_indexingType.offset() & (sizeof(int32_t) - 1)));
RELEASE_ASSERT(JSCell_indexingType.offset() + 1 == JSCell_typeInfoType.offset());
RELEASE_ASSERT(JSCell_indexingType.offset() + 2 == JSCell_typeInfoFlags.offset());
- RELEASE_ASSERT(JSCell_indexingType.offset() + 3 == JSCell_gcData.offset());
+ RELEASE_ASSERT(JSCell_indexingType.offset() + 3 == JSCell_cellState.offset());
JSCell_indexingType.changeParent(&JSCell_usefulBytes);
JSCell_typeInfoType.changeParent(&JSCell_usefulBytes);
JSCell_typeInfoFlags.changeParent(&JSCell_usefulBytes);
- JSCell_gcData.changeParent(&JSCell_usefulBytes);
+ JSCell_cellState.changeParent(&JSCell_usefulBytes);
root.m_tbaaMetadata = mdNode(m_context, mdString(m_context, root.m_heapName));
Modified: trunk/Source/_javascript_Core/ftl/FTLAbstractHeapRepository.h (190568 => 190569)
--- trunk/Source/_javascript_Core/ftl/FTLAbstractHeapRepository.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/ftl/FTLAbstractHeapRepository.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -52,12 +52,12 @@
macro(JSArrayBufferView_length, JSArrayBufferView::offsetOfLength()) \
macro(JSArrayBufferView_mode, JSArrayBufferView::offsetOfMode()) \
macro(JSArrayBufferView_vector, JSArrayBufferView::offsetOfVector()) \
+ macro(JSCell_cellState, JSCell::cellStateOffset()) \
+ macro(JSCell_indexingType, JSCell::indexingTypeOffset()) \
macro(JSCell_structureID, JSCell::structureIDOffset()) \
- macro(JSCell_usefulBytes, JSCell::indexingTypeOffset()) \
macro(JSCell_typeInfoFlags, JSCell::typeInfoFlagsOffset()) \
macro(JSCell_typeInfoType, JSCell::typeInfoTypeOffset()) \
- macro(JSCell_indexingType, JSCell::indexingTypeOffset()) \
- macro(JSCell_gcData, JSCell::gcDataOffset()) \
+ macro(JSCell_usefulBytes, JSCell::indexingTypeOffset()) \
macro(JSFunction_executable, JSFunction::offsetOfExecutable()) \
macro(JSFunction_scope, JSFunction::offsetOfScopeChain()) \
macro(JSFunction_rareData, JSFunction::offsetOfRareData()) \
Modified: trunk/Source/_javascript_Core/ftl/FTLLowerDFGToLLVM.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/ftl/FTLLowerDFGToLLVM.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/ftl/FTLLowerDFGToLLVM.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -8260,9 +8260,9 @@
return m_graph.masqueradesAsUndefinedWatchpointIsStillValid(m_node->origin.semantic);
}
- LValue loadMarkByte(LValue base)
+ LValue loadCellState(LValue base)
{
- return m_out.load8(base, m_heaps.JSCell_gcData);
+ return m_out.load8(base, m_heaps.JSCell_cellState);
}
void emitStoreBarrier(LValue base)
@@ -8274,7 +8274,7 @@
// Check the mark byte.
m_out.branch(
- m_out.notZero8(loadMarkByte(base)), usually(continuation), rarely(isMarkedAndNotRemembered));
+ m_out.notZero8(loadCellState(base)), usually(continuation), rarely(isMarkedAndNotRemembered));
// Append to the write barrier buffer.
LBasicBlock lastNext = m_out.appendTo(isMarkedAndNotRemembered, bufferHasSpace);
Added: trunk/Source/_javascript_Core/heap/CellState.h (0 => 190569)
--- trunk/Source/_javascript_Core/heap/CellState.h (rev 0)
+++ trunk/Source/_javascript_Core/heap/CellState.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -0,0 +1,60 @@
+/*
+ * Copyright (C) 2015 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef CellState_h
+#define CellState_h
+
+namespace JSC {
+
+enum class CellState : uint8_t {
+ // The object is black as far as this GC is concerned. When not in GC, this just means that it's an
+ // old gen object. Note that we deliberately arrange OldBlack to be zero, so that the store barrier on
+ // a target object "from" is just:
+ //
+ // if (!from->cellState())
+ // slowPath(from);
+ //
+ // There is a bunch of code in the LLInt and JITs that rely on this being the case. You'd have to
+ // change a lot of code if you ever wanted the store barrier to be anything but a non-zero check on
+ // cellState.
+ OldBlack = 0,
+
+ // The object is in eden. During GC, this means that the object has not been marked yet.
+ NewWhite = 1,
+
+ // The object is grey - i.e. it will be scanned - but it either belongs to old gen (if this is eden
+ // GC) or it is grey a second time in this current GC (because a concurrent store barrier requested
+ // re-greying).
+ OldGrey = 2,
+
+ // The object is grey - i.e. it will be scanned - and this is the first time in this GC that we are
+ // going to scan it. If this is an eden GC, this also means that the object is in eden.
+ NewGrey = 3
+};
+
+} // namespace JSC
+
+#endif // CellState_h
+
Modified: trunk/Source/_javascript_Core/heap/CodeBlockSet.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/CodeBlockSet.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/CodeBlockSet.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013, 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -151,7 +151,7 @@
if (verbose)
dataLog("Remembering ", m_currentlyExecuting.size(), " code blocks.\n");
for (const RefPtr<CodeBlock>& codeBlock : m_currentlyExecuting)
- heap->addToRememberedSet(codeBlock->ownerExecutable());
+ heap->writeBarrier(codeBlock->ownerExecutable());
// It's safe to clear these RefPtr sets because we won't delete the CodeBlocks
// in them until the next GC, and we'll recompute them at that time.
Modified: trunk/Source/_javascript_Core/heap/CopiedBlock.h (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/CopiedBlock.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/CopiedBlock.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2011 Apple Inc. All rights reserved.
+ * Copyright (C) 2011, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -54,7 +54,6 @@
void didPromote();
unsigned liveBytes();
- bool shouldReportLiveBytes(LockHolder&, JSCell* owner);
void reportLiveBytes(LockHolder&, JSCell*, CopyToken, unsigned);
void reportLiveBytesDuringCopying(unsigned);
void didSurviveGC();
Modified: trunk/Source/_javascript_Core/heap/CopiedBlockInlines.h (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/CopiedBlockInlines.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/CopiedBlockInlines.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2012, 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2012, 2013, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -33,16 +33,6 @@
namespace JSC {
-inline bool CopiedBlock::shouldReportLiveBytes(LockHolder&, JSCell* owner)
-{
- // We want to add to live bytes if the owner isn't part of the remembered set or
- // if this block was allocated during the last cycle.
- // If we always added live bytes we would double count for elements in the remembered
- // set across collections.
- // If we didn't always add live bytes to new blocks, we'd get too few.
- return !Heap::isRemembered(owner) || !m_isOld;
-}
-
inline void CopiedBlock::reportLiveBytes(LockHolder&, JSCell* owner, CopyToken token, unsigned bytes)
{
checkConsistency();
Modified: trunk/Source/_javascript_Core/heap/GCLogging.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/GCLogging.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/GCLogging.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -91,7 +91,7 @@
for (const JSCell* cell : m_savedMarkStack) {
m_slotVisitor.markStack().append(cell);
- const_cast<JSCell*>(cell)->setRemembered(true);
+ cell->setCellState(CellState::OldGrey);
}
}
Modified: trunk/Source/_javascript_Core/heap/Heap.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/Heap.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/Heap.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -596,7 +596,6 @@
// the liveness of the rest of the object graph.
visitWeakHandles(heapRootVisitor);
- clearRememberedSet(rememberedSet);
{
std::lock_guard<Lock> lock(m_markingMutex);
m_parallelMarkersShouldExit = true;
@@ -852,13 +851,6 @@
}
}
-void Heap::clearRememberedSet(Vector<const JSCell*>& rememberedSet)
-{
- GCPHASE(ClearRememberedSet);
- for (auto* cell : rememberedSet)
- const_cast<JSCell*>(cell)->setRemembered(false);
-}
-
void Heap::updateObjectCounts(double gcStartTime)
{
GCCOUNTER(VisitedValueCount, m_slotVisitor.visitCount());
@@ -1008,9 +1000,13 @@
{
ASSERT(cell);
ASSERT(!Options::enableConcurrentJIT() || !isCompilationThread());
- if (isRemembered(cell))
- return;
- const_cast<JSCell*>(cell)->setRemembered(true);
+ ASSERT(cell->cellState() == CellState::OldBlack);
+ // Indicate that this object is grey and that it's one of the following:
+ // - A re-greyed object during a concurrent collection.
+ // - An old remembered object.
+ // "OldGrey" doesn't tell us which of these things is true, but we usually treat the two cases the
+ // same.
+ cell->setCellState(CellState::OldGrey);
m_slotVisitor.appendToMarkStack(const_cast<JSCell*>(cell));
}
Modified: trunk/Source/_javascript_Core/heap/Heap.h (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/Heap.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/Heap.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -101,9 +101,7 @@
static bool isMarked(const void*);
static bool testAndSetMarked(const void*);
static void setMarked(const void*);
- static bool isRemembered(const void*);
- JS_EXPORT_PRIVATE void addToRememberedSet(const JSCell*);
static bool isWriteBarrierEnabled();
void writeBarrier(const JSCell*);
void writeBarrier(const JSCell*, JSValue);
@@ -167,7 +165,7 @@
// call both of these functions: Calling only one may trigger catastropic
// memory growth.
void reportExtraMemoryAllocated(size_t);
- void reportExtraMemoryVisited(JSCell*, size_t);
+ void reportExtraMemoryVisited(CellState cellStateBeforeVisiting, size_t);
// Use this API to report non-GC memory if you can't use the better API above.
void deprecatedReportExtraMemory(size_t);
@@ -306,7 +304,6 @@
void traceCodeBlocksAndJITStubRoutines();
void converge();
void visitWeakHandles(HeapRootVisitor&);
- void clearRememberedSet(Vector<const JSCell*>&);
void updateObjectCounts(double gcStartTime);
void resetVisitors();
@@ -323,6 +320,7 @@
void finalizeUnconditionalFinalizers();
void clearUnmarkedExecutables();
void deleteUnmarkedCompiledCode();
+ JS_EXPORT_PRIVATE void addToRememberedSet(const JSCell*);
void updateAllocationLimits();
void didFinishCollection(double gcStartTime);
void resumeCompilerThreads();
Modified: trunk/Source/_javascript_Core/heap/HeapInlines.h (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/HeapInlines.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/HeapInlines.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -70,14 +70,6 @@
return MarkedBlock::blockFor(cell)->isLiveCell(cell);
}
-inline bool Heap::isRemembered(const void* ptr)
-{
- const JSCell* cell = static_cast<const JSCell*>(ptr);
- ASSERT(cell);
- ASSERT(!Options::enableConcurrentJIT() || !isCompilationThread());
- return cell->isRemembered();
-}
-
inline bool Heap::isMarked(const void* cell)
{
return MarkedBlock::blockFor(cell)->isMarked(cell);
@@ -113,25 +105,18 @@
#if ENABLE(WRITE_BARRIER_PROFILING)
WriteBarrierCounters::countWriteBarrier();
#endif
- if (!from || !from->isMarked()) {
- ASSERT(!from || !isMarked(from));
+ if (!from || from->cellState() != CellState::OldBlack)
return;
- }
- if (!to || to->isMarked()) {
- ASSERT(!to || isMarked(to));
+ if (!to || to->cellState() != CellState::NewWhite)
return;
- }
addToRememberedSet(from);
}
inline void Heap::writeBarrier(const JSCell* from)
{
ASSERT_GC_OBJECT_LOOKS_VALID(const_cast<JSCell*>(from));
- if (!from || !from->isMarked()) {
- ASSERT(!from || !isMarked(from));
+ if (!from || from->cellState() != CellState::OldBlack)
return;
- }
- ASSERT(isMarked(from));
addToRememberedSet(from);
}
@@ -141,10 +126,10 @@
reportExtraMemoryAllocatedSlowCase(size);
}
-inline void Heap::reportExtraMemoryVisited(JSCell* owner, size_t size)
+inline void Heap::reportExtraMemoryVisited(CellState dataBeforeVisiting, size_t size)
{
// We don't want to double-count the extra memory that was reported in previous collections.
- if (operationInProgress() == EdenCollection && Heap::isRemembered(owner))
+ if (operationInProgress() == EdenCollection && dataBeforeVisiting == CellState::OldGrey)
return;
size_t* counter = &m_extraMemorySize;
Modified: trunk/Source/_javascript_Core/heap/SlotVisitor.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/SlotVisitor.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/SlotVisitor.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -139,7 +139,11 @@
return;
}
- cell->setMarked();
+ // Indicate that the object is grey and that:
+ // In case of concurrent GC: it's the first time it is grey in this GC cycle.
+ // In case of eden collection: it's a new object that became grey rather than an old remembered object.
+ cell->setCellState(CellState::NewGrey);
+
appendToMarkStack(cell);
}
@@ -153,26 +157,29 @@
m_stack.append(cell);
}
-ALWAYS_INLINE static void visitChildren(SlotVisitor& visitor, const JSCell* cell)
+ALWAYS_INLINE void SlotVisitor::visitChildren(const JSCell* cell)
{
ASSERT(Heap::isMarked(cell));
+
+ m_currentObjectCellStateBeforeVisiting = cell->cellState();
+ cell->setCellState(CellState::OldBlack);
if (isJSString(cell)) {
- JSString::visitChildren(const_cast<JSCell*>(cell), visitor);
+ JSString::visitChildren(const_cast<JSCell*>(cell), *this);
return;
}
if (isJSFinalObject(cell)) {
- JSFinalObject::visitChildren(const_cast<JSCell*>(cell), visitor);
+ JSFinalObject::visitChildren(const_cast<JSCell*>(cell), *this);
return;
}
if (isJSArray(cell)) {
- JSArray::visitChildren(const_cast<JSCell*>(cell), visitor);
+ JSArray::visitChildren(const_cast<JSCell*>(cell), *this);
return;
}
- cell->methodTable()->visitChildren(const_cast<JSCell*>(cell), visitor);
+ cell->methodTable()->visitChildren(const_cast<JSCell*>(cell), *this);
}
void SlotVisitor::donateKnownParallel()
@@ -208,7 +215,7 @@
while (!m_stack.isEmpty()) {
m_stack.refill();
for (unsigned countdown = Options::minimumNumberOfScansBetweenRebalance(); m_stack.canRemoveLast() && countdown--;)
- visitChildren(*this, m_stack.removeLast());
+ visitChildren(m_stack.removeLast());
donateKnownParallel();
}
@@ -364,7 +371,14 @@
ASSERT(heap()->m_storageSpace.contains(block));
LockHolder locker(&block->workListLock());
- if (heap()->operationInProgress() == FullCollection || block->shouldReportLiveBytes(locker, owner)) {
+ // We always report live bytes, except if during an eden collection we see an old object pointing to an
+ // old backing store and the old object is being marked because of the remembered set. Note that if we
+ // ask the object itself, it will always tell us that it's an old black object - because even during an
+ // eden collection we have already indicated that the object is old. That's why we use the
+ // SlotVisitor's cache of the object's old state.
+ if (heap()->operationInProgress() == FullCollection
+ || !block->isOld()
+ || m_currentObjectCellStateBeforeVisiting != CellState::OldGrey) {
m_bytesCopied += bytes;
block->reportLiveBytes(locker, owner, token, bytes);
}
Modified: trunk/Source/_javascript_Core/heap/SlotVisitor.h (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/SlotVisitor.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/SlotVisitor.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2011, 2012, 2013, 2015 Apple Inc. All rights reserved.
+ * Copyright (C) 2011-2013, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -26,6 +26,7 @@
#ifndef SlotVisitor_h
#define SlotVisitor_h
+#include "CellState.h"
#include "CopyToken.h"
#include "HandleTypes.h"
#include "MarkStack.h"
@@ -103,7 +104,7 @@
void copyLater(JSCell*, CopyToken, void*, size_t);
- void reportExtraMemoryVisited(JSCell* owner, size_t);
+ void reportExtraMemoryVisited(size_t);
void addWeakReferenceHarvester(WeakReferenceHarvester*);
void addUnconditionalFinalizer(UnconditionalFinalizer*);
@@ -121,6 +122,8 @@
JS_EXPORT_PRIVATE void mergeOpaqueRoots();
void mergeOpaqueRootsIfNecessary();
void mergeOpaqueRootsIfProfitable();
+
+ void visitChildren(const JSCell*);
void donateKnownParallel();
@@ -134,6 +137,8 @@
Heap& m_heap;
+ CellState m_currentObjectCellStateBeforeVisiting { CellState::NewWhite };
+
public:
#if !ASSERT_DISABLED
bool m_isCheckingForDefaultMarkViolation;
Modified: trunk/Source/_javascript_Core/heap/SlotVisitorInlines.h (190568 => 190569)
--- trunk/Source/_javascript_Core/heap/SlotVisitorInlines.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/heap/SlotVisitorInlines.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -92,9 +92,9 @@
m_heap.m_unconditionalFinalizers.addThreadSafe(unconditionalFinalizer);
}
-inline void SlotVisitor::reportExtraMemoryVisited(JSCell* owner, size_t size)
+inline void SlotVisitor::reportExtraMemoryVisited(size_t size)
{
- heap()->reportExtraMemoryVisited(owner, size);
+ heap()->reportExtraMemoryVisited(m_currentObjectCellStateBeforeVisiting, size);
}
inline Heap* SlotVisitor::heap() const
Modified: trunk/Source/_javascript_Core/jit/AssemblyHelpers.h (190568 => 190569)
--- trunk/Source/_javascript_Core/jit/AssemblyHelpers.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/jit/AssemblyHelpers.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1165,12 +1165,12 @@
Jump jumpIfIsRememberedOrInEden(GPRReg cell)
{
- return branchTest8(MacroAssembler::NonZero, MacroAssembler::Address(cell, JSCell::gcDataOffset()));
+ return branchTest8(MacroAssembler::NonZero, MacroAssembler::Address(cell, JSCell::cellStateOffset()));
}
Jump jumpIfIsRememberedOrInEden(JSCell* cell)
{
- uint8_t* address = reinterpret_cast<uint8_t*>(cell) + JSCell::gcDataOffset();
+ uint8_t* address = reinterpret_cast<uint8_t*>(cell) + JSCell::cellStateOffset();
return branchTest8(MacroAssembler::NonZero, MacroAssembler::AbsoluteAddress(address));
}
Modified: trunk/Source/_javascript_Core/llint/LowLevelInterpreter.asm (190568 => 190569)
--- trunk/Source/_javascript_Core/llint/LowLevelInterpreter.asm 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/llint/LowLevelInterpreter.asm 2015-10-05 19:35:32 UTC (rev 190569)
@@ -799,7 +799,7 @@
end
macro skipIfIsRememberedOrInEden(cell, scratch1, scratch2, continuation)
- loadb JSCell::m_gcData[cell], scratch1
+ loadb JSCell::m_cellState[cell], scratch1
continuation(scratch1)
end
Modified: trunk/Source/_javascript_Core/llint/LowLevelInterpreter32_64.asm (190568 => 190569)
--- trunk/Source/_javascript_Core/llint/LowLevelInterpreter32_64.asm 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/llint/LowLevelInterpreter32_64.asm 2015-10-05 19:35:32 UTC (rev 190569)
@@ -497,8 +497,8 @@
loadisFromInstruction(cellOperand, t1)
loadConstantOrVariablePayload(t1, CellTag, t2, .writeBarrierDone)
skipIfIsRememberedOrInEden(t2, t1, t3,
- macro(gcData)
- btbnz gcData, .writeBarrierDone
+ macro(cellState)
+ btbnz cellState, .writeBarrierDone
push cfr, PC
# We make two extra slots because cCall2 will poke.
subp 8, sp
Modified: trunk/Source/_javascript_Core/llint/LowLevelInterpreter64.asm (190568 => 190569)
--- trunk/Source/_javascript_Core/llint/LowLevelInterpreter64.asm 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/llint/LowLevelInterpreter64.asm 2015-10-05 19:35:32 UTC (rev 190569)
@@ -401,8 +401,8 @@
loadisFromInstruction(cellOperand, t1)
loadConstantOrVariableCell(t1, t2, .writeBarrierDone)
skipIfIsRememberedOrInEden(t2, t1, t3,
- macro(gcData)
- btbnz gcData, .writeBarrierDone
+ macro(cellState)
+ btbnz cellState, .writeBarrierDone
push PB, PC
move t2, a1 # t2 can be a0 (not on 64 bits, but better safe than sorry)
move cfr, a0
Modified: trunk/Source/_javascript_Core/runtime/JSCell.h (190568 => 190569)
--- trunk/Source/_javascript_Core/runtime/JSCell.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/runtime/JSCell.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,7 +1,7 @@
/*
* Copyright (C) 1999-2001 Harri Porten ([email protected])
* Copyright (C) 2001 Peter Kelly ([email protected])
- * Copyright (C) 2003, 2004, 2005, 2007, 2008, 2009 Apple Inc. All rights reserved.
+ * Copyright (C) 2003, 2004, 2005, 2007, 2008, 2009, 2015 Apple Inc. All rights reserved.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
@@ -24,6 +24,7 @@
#define JSCell_h
#include "CallData.h"
+#include "CellState.h"
#include "ConstructData.h"
#include "EnumerationMode.h"
#include "Heap.h"
@@ -154,37 +155,13 @@
static bool canUseFastGetOwnProperty(const Structure&);
JSValue fastGetOwnProperty(VM&, Structure&, PropertyName);
- enum GCData : uint8_t {
- Marked = 0, // The object has survived a GC and is in the old gen.
- NotMarked = 1, // The object is new and in the eden gen.
- MarkedAndRemembered = 2, // The object is in the GC's remembered set.
+ // The recommended idiom for using cellState() is to switch on it or perform an == comparison on it
+ // directly. We deliberately avoid helpers for this, because we want transparency about how the various
+ // CellState values influences our various algorithms.
+ CellState cellState() const { return m_cellState; }
+
+ void setCellState(CellState data) const { const_cast<JSCell*>(this)->m_cellState = data; }
- // The object being in the GC's remembered set implies that it is also
- // Marked. This is because objects are only added to the remembered sets
- // by write barriers, and write barriers are only interested in old gen
- // objects that point to potential eden gen objects.
- };
-
- void setMarked() { m_gcData = Marked; }
- void setRemembered(bool remembered)
- {
- ASSERT(m_gcData == (remembered ? Marked : MarkedAndRemembered));
- m_gcData = remembered ? MarkedAndRemembered : Marked;
- }
- bool isMarked() const
- {
- switch (m_gcData) {
- case Marked:
- case MarkedAndRemembered:
- return true;
- case NotMarked:
- return false;
- }
- RELEASE_ASSERT_NOT_REACHED();
- return false;
- }
- bool isRemembered() const { return m_gcData == MarkedAndRemembered; }
-
static ptrdiff_t structureIDOffset()
{
return OBJECT_OFFSETOF(JSCell, m_structureID);
@@ -205,9 +182,9 @@
return OBJECT_OFFSETOF(JSCell, m_indexingType);
}
- static ptrdiff_t gcDataOffset()
+ static ptrdiff_t cellStateOffset()
{
- return OBJECT_OFFSETOF(JSCell, m_gcData);
+ return OBJECT_OFFSETOF(JSCell, m_cellState);
}
static const TypedArrayType TypedArrayStorageType = NotTypedArray;
@@ -241,7 +218,7 @@
IndexingType m_indexingType;
JSType m_type;
TypeInfo::InlineTypeFlags m_flags;
- uint8_t m_gcData;
+ CellState m_cellState;
};
template<typename To, typename From>
Modified: trunk/Source/_javascript_Core/runtime/JSCellInlines.h (190568 => 190569)
--- trunk/Source/_javascript_Core/runtime/JSCellInlines.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/runtime/JSCellInlines.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2012, 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2012, 2013, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -41,7 +41,7 @@
namespace JSC {
inline JSCell::JSCell(CreatingEarlyCellTag)
- : m_gcData(NotMarked)
+ : m_cellState(CellState::NewWhite)
{
ASSERT(!isCompilationThread());
}
@@ -51,7 +51,7 @@
, m_indexingType(structure->indexingType())
, m_type(structure->typeInfo().type())
, m_flags(structure->typeInfo().inlineTypeFlags())
- , m_gcData(NotMarked)
+ , m_cellState(CellState::NewWhite)
{
ASSERT(!isCompilationThread());
}
Modified: trunk/Source/_javascript_Core/runtime/JSGenericTypedArrayViewInlines.h (190568 => 190569)
--- trunk/Source/_javascript_Core/runtime/JSGenericTypedArrayViewInlines.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/runtime/JSGenericTypedArrayViewInlines.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -446,7 +446,7 @@
}
case OversizeTypedArray: {
- visitor.reportExtraMemoryVisited(thisObject, thisObject->byteSize());
+ visitor.reportExtraMemoryVisited(thisObject->byteSize());
break;
}
Modified: trunk/Source/_javascript_Core/runtime/JSObject.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/runtime/JSObject.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/runtime/JSObject.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -219,7 +219,7 @@
{
JSObject* thisObject = jsCast<JSObject*>(cell);
ASSERT_GC_OBJECT_INHERITS(thisObject, info());
-
+
if (token != ButterflyCopyToken)
return;
Modified: trunk/Source/_javascript_Core/runtime/JSString.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/runtime/JSString.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/runtime/JSString.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,7 +1,7 @@
/*
* Copyright (C) 1999-2002 Harri Porten ([email protected])
* Copyright (C) 2001 Peter Kelly ([email protected])
- * Copyright (C) 2004, 2007, 2008 Apple Inc. All rights reserved.
+ * Copyright (C) 2004, 2007, 2008, 2015 Apple Inc. All rights reserved.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
@@ -77,7 +77,7 @@
else {
StringImpl* impl = thisObject->m_value.impl();
ASSERT(impl);
- visitor.reportExtraMemoryVisited(thisObject, impl->costDuringGC());
+ visitor.reportExtraMemoryVisited(impl->costDuringGC());
}
}
Modified: trunk/Source/_javascript_Core/runtime/StructureIDBlob.h (190568 => 190569)
--- trunk/Source/_javascript_Core/runtime/StructureIDBlob.h 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/runtime/StructureIDBlob.h 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2014 Apple Inc. All rights reserved.
+ * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -26,6 +26,7 @@
#ifndef StructureIDBlob_h
#define StructureIDBlob_h
+#include "CellState.h"
#include "IndexingType.h"
#include "JSTypeInfo.h"
#include "StructureIDTable.h"
@@ -46,7 +47,7 @@
u.fields.indexingType = indexingType;
u.fields.type = typeInfo.type();
u.fields.inlineTypeFlags = typeInfo.inlineTypeFlags();
- u.fields.defaultGCData = JSCell::NotMarked;
+ u.fields.defaultCellState = CellState::NewWhite;
}
void operator=(const StructureIDBlob& other) { u.doubleWord = other.u.doubleWord; }
@@ -79,7 +80,7 @@
IndexingType indexingType;
JSType type;
TypeInfo::InlineTypeFlags inlineTypeFlags;
- JSCell::GCData defaultGCData;
+ CellState defaultCellState;
} fields;
struct {
int32_t word1;
Modified: trunk/Source/_javascript_Core/runtime/WeakMapData.cpp (190568 => 190569)
--- trunk/Source/_javascript_Core/runtime/WeakMapData.cpp 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/_javascript_Core/runtime/WeakMapData.cpp 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013, 2015 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -64,7 +64,7 @@
// Rough approximation of the external storage needed for the hashtable.
// This isn't exact, but it is close enough, and proportional to the actual
// external mermory usage.
- visitor.reportExtraMemoryVisited(thisObj, thisObj->m_map.capacity() * (sizeof(JSObject*) + sizeof(WriteBarrier<Unknown>)));
+ visitor.reportExtraMemoryVisited(thisObj->m_map.capacity() * (sizeof(JSObject*) + sizeof(WriteBarrier<Unknown>)));
}
void WeakMapData::set(VM& vm, JSObject* key, JSValue value)
Added: trunk/Source/_javascript_Core/tests/stress/basic-eden-gc-test.js (0 => 190569)
--- trunk/Source/_javascript_Core/tests/stress/basic-eden-gc-test.js (rev 0)
+++ trunk/Source/_javascript_Core/tests/stress/basic-eden-gc-test.js 2015-10-05 19:35:32 UTC (rev 190569)
@@ -0,0 +1,4 @@
+var o = {f:42};
+edenGC();
+var p = {f:42};
+edenGC();
Modified: trunk/Source/WebCore/ChangeLog (190568 => 190569)
--- trunk/Source/WebCore/ChangeLog 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/WebCore/ChangeLog 2015-10-05 19:35:32 UTC (rev 190569)
@@ -1,3 +1,15 @@
+2015-10-03 Filip Pizlo <[email protected]>
+
+ Allow an object's marking state to track The Three Colors
+ https://bugs.webkit.org/show_bug.cgi?id=149654
+
+ Reviewed by Geoffrey Garen.
+
+ No new tests because no new behavior.
+
+ * bindings/scripts/CodeGeneratorJS.pm:
+ (GenerateImplementation):
+
2015-10-05 Katlyn Graff <[email protected]>
Update setImageSmoothingQuality for additional reviews.
Modified: trunk/Source/WebCore/bindings/scripts/CodeGeneratorJS.pm (190568 => 190569)
--- trunk/Source/WebCore/bindings/scripts/CodeGeneratorJS.pm 2015-10-05 18:49:54 UTC (rev 190568)
+++ trunk/Source/WebCore/bindings/scripts/CodeGeneratorJS.pm 2015-10-05 19:35:32 UTC (rev 190569)
@@ -2989,7 +2989,7 @@
}
push(@implContent, " thisObject->visitAdditionalChildren(visitor);\n") if $interface->extendedAttributes->{"JSCustomMarkFunction"};
if ($interface->extendedAttributes->{"ReportExtraMemoryCost"}) {
- push(@implContent, " visitor.reportExtraMemoryVisited(cell, thisObject->impl().memoryCost());\n");
+ push(@implContent, " visitor.reportExtraMemoryVisited(thisObject->impl().memoryCost());\n");
}
if ($numCachedAttributes > 0) {
foreach (@{$interface->attributes}) {
