Title: [196922] releases/WebKitGTK/webkit-2.10/Source/bmalloc
Revision
196922
Author
[email protected]
Date
2016-02-22 02:19:05 -0800 (Mon, 22 Feb 2016)

Log Message

Merge r196421, r196424, r196536: bmalloc: large aligned allocations will put 1 or 2 free object on free list without merging with free neighbors
https://bugs.webkit.org/show_bug.cgi?id=154091

Reviewed by Geoffrey Garen.

If we split off any unused free object in the aligned version of Heap::allocateLarge(), we merge them with
free neighbors before putting them back on the free list.  Added helpers to verify that when we
add LargeObjects to the free list their neighbors are allocated.

* bmalloc/Heap.cpp:
(bmalloc::Heap::allocateLarge): Deleted private helper version and rolled it into the two the
two public versions of allocateLarge().
* bmalloc/Heap.h:
* bmalloc/LargeObject.h:
(bmalloc::LargeObject::prevIsAllocated): New helper.
(bmalloc::LargeObject::nextIsAllocated): New helper.
(bmalloc::LargeObject::merge): Check that the merge object has allocated neighbors.

Unreviewed build fix after r196421.

Removed BASSERTs that are firing to eliminate Debug build crashes.  I'll debug locally and
enable or alter after the issue is understood.

* bmalloc/LargeObject.h:
(bmalloc::LargeObject::merge): Removed BASSERTs that are firing.

BASSERTs added in r196421 are causing debug test failures
https://bugs.webkit.org/show_bug.cgi?id=154113

Reviewed by Geoffrey Garen.

In VMHeap::deallocateLargeObject(), we drop the lock to deallocate the physical pages.
If the scavenger thread is running at the same time a synchronous call to scavenge()
comes in, we could call VMHeap::deallocateLargeObject() for an adjacent object while the
lock in the other thread is dropped.  We fix this by checking for adjacent objects we
can merge with and loop if we have one.

* bmalloc/FreeList.h:
(bmalloc::FreeList::push): Added BASSERT to catch adding unmerged free objects
* bmalloc/Heap.cpp:
(bmalloc::Heap::allocateLarge): Changed to use nextCanMerge().
* bmalloc/LargeObject.h:
(bmalloc::LargeObject::prevCanMerge): Repurposed prevIsAllocated.
(bmalloc::LargeObject::nextCanMerge): Repurposed nextIsAllocated.
(bmalloc::LargeObject::prevIsAllocated): Deleted.
(bmalloc::LargeObject::nextIsAllocated): Deleted.
* bmalloc/VMHeap.h:
(bmalloc::VMHeap::allocateLargeObject): Moved adding the extra object back to the free list
to after we set the object we'll return as being allocated.
(bmalloc::VMHeap::deallocateLargeObject):

Modified Paths

Diff

Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/ChangeLog (196921 => 196922)


--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/ChangeLog	2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/ChangeLog	2016-02-22 10:19:05 UTC (rev 196922)
@@ -1,3 +1,60 @@
+2016-02-12  Michael Saboff  <[email protected]>
+
+        BASSERTs added in r196421 are causing debug test failures
+        https://bugs.webkit.org/show_bug.cgi?id=154113
+
+        Reviewed by Geoffrey Garen.
+
+        In VMHeap::deallocateLargeObject(), we drop the lock to deallocate the physical pages.
+        If the scavenger thread is running at the same time a synchronous call to scavenge()
+        comes in, we could call VMHeap::deallocateLargeObject() for an adjacent object while the
+        lock in the other thread is dropped.  We fix this by checking for adjacent objects we
+        can merge with and loop if we have one.
+
+        * bmalloc/FreeList.h:
+        (bmalloc::FreeList::push): Added BASSERT to catch adding unmerged free objects
+        * bmalloc/Heap.cpp:
+        (bmalloc::Heap::allocateLarge): Changed to use nextCanMerge().
+        * bmalloc/LargeObject.h:
+        (bmalloc::LargeObject::prevCanMerge): Repurposed prevIsAllocated.
+        (bmalloc::LargeObject::nextCanMerge): Repurposed nextIsAllocated.
+        (bmalloc::LargeObject::prevIsAllocated): Deleted.
+        (bmalloc::LargeObject::nextIsAllocated): Deleted.
+        * bmalloc/VMHeap.h:
+        (bmalloc::VMHeap::allocateLargeObject): Moved adding the extra object back to the free list
+        to after we set the object we'll return as being allocated.
+        (bmalloc::VMHeap::deallocateLargeObject):
+
+2016-02-11  Michael Saboff  <[email protected]>
+
+        Unreviewed build fix after r196421.
+
+        Removed BASSERTs that are firing to eliminate Debug build crashes.  I'll debug locally and
+        enable or alter after the issue is understood.
+
+        * bmalloc/LargeObject.h:
+        (bmalloc::LargeObject::merge): Removed BASSERTs that are firing.
+
+2016-02-11  Michael Saboff  <[email protected]>
+
+        bmalloc: large aligned allocations will put 1 or 2 free object on free list without merging with free neighbors
+        https://bugs.webkit.org/show_bug.cgi?id=154091
+
+        Reviewed by Geoffrey Garen.
+
+        If we split off any unused free object in the aligned version of Heap::allocateLarge(), we merge them with
+        free neighbors before putting them back on the free list.  Added helpers to verify that when we
+        add LargeObjects to the free list their neighbors are allocated.
+
+        * bmalloc/Heap.cpp:
+        (bmalloc::Heap::allocateLarge): Deleted private helper version and rolled it into the two the
+        two public versions of allocateLarge().
+        * bmalloc/Heap.h:
+        * bmalloc/LargeObject.h:
+        (bmalloc::LargeObject::prevIsAllocated): New helper.
+        (bmalloc::LargeObject::nextIsAllocated): New helper.
+        (bmalloc::LargeObject::merge): Check that the merge object has allocated neighbors.
+
 2015-12-03  Michael Saboff  <[email protected]>
 
         bmalloc: extra large allocations could be more efficient

Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/FreeList.h (196921 => 196922)


--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/FreeList.h	2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/FreeList.h	2016-02-22 10:19:05 UTC (rev 196922)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
+ * Copyright (C) 2014-2016 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -60,6 +60,8 @@
 inline void FreeList::push(Owner owner, const LargeObject& largeObject)
 {
     BASSERT(largeObject.isFree());
+    BASSERT(!largeObject.prevCanMerge());
+    BASSERT(!largeObject.nextCanMerge());
     if (m_vector.size() == m_limit) {
         removeInvalidAndDuplicateEntries(owner);
         m_limit = std::max(m_vector.size() * freeListGrowFactor, freeListSearchDepth);

Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.cpp (196921 => 196922)


--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.cpp	2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.cpp	2016-02-22 10:19:05 UTC (rev 196922)
@@ -346,22 +346,8 @@
     lock.lock();
 }
 
-void* Heap::allocateLarge(std::lock_guard<StaticMutex>&, LargeObject& largeObject, size_t size)
+void* Heap::allocateLarge(std::lock_guard<StaticMutex>&, size_t size)
 {
-    BASSERT(largeObject.isFree());
-
-    if (largeObject.size() - size > largeMin) {
-        std::pair<LargeObject, LargeObject> split = largeObject.split(size);
-        largeObject = split.first;
-        m_largeObjects.insert(split.second);
-    }
-
-    largeObject.setFree(false);
-    return largeObject.begin();
-}
-
-void* Heap::allocateLarge(std::lock_guard<StaticMutex>& lock, size_t size)
-{
     BASSERT(size <= largeMax);
     BASSERT(size >= largeMin);
     BASSERT(size == roundUpToMultipleOf<largeAlignment>(size));
@@ -372,10 +358,27 @@
         largeObject = m_vmHeap.allocateLargeObject(size);
     }
 
-    return allocateLarge(lock, largeObject, size);
+    BASSERT(largeObject.isFree());
+
+    LargeObject nextLargeObject;
+
+    if (largeObject.size() - size > largeMin) {
+        std::pair<LargeObject, LargeObject> split = largeObject.split(size);
+        largeObject = split.first;
+        nextLargeObject = split.second;
+    }
+    
+    largeObject.setFree(false);
+    
+    if (nextLargeObject) {
+        BASSERT(!nextLargeObject.nextCanMerge());
+        m_largeObjects.insert(nextLargeObject);
+    }
+    
+    return largeObject.begin();
 }
 
-void* Heap::allocateLarge(std::lock_guard<StaticMutex>& lock, size_t alignment, size_t size, size_t unalignedSize)
+void* Heap::allocateLarge(std::lock_guard<StaticMutex>&, size_t alignment, size_t size, size_t unalignedSize)
 {
     BASSERT(size <= largeMax);
     BASSERT(size >= largeMin);
@@ -393,15 +396,38 @@
         largeObject = m_vmHeap.allocateLargeObject(alignment, size, unalignedSize);
     }
 
+    LargeObject prevLargeObject;
+    LargeObject nextLargeObject;
+
     size_t alignmentMask = alignment - 1;
     if (test(largeObject.begin(), alignmentMask)) {
         size_t prefixSize = roundUpToMultipleOf(alignment, largeObject.begin() + largeMin) - largeObject.begin();
         std::pair<LargeObject, LargeObject> pair = largeObject.split(prefixSize);
-        m_largeObjects.insert(pair.first);
+        prevLargeObject = pair.first;
         largeObject = pair.second;
     }
 
-    return allocateLarge(lock, largeObject, size);
+    BASSERT(largeObject.isFree());
+    
+    if (largeObject.size() - size > largeMin) {
+        std::pair<LargeObject, LargeObject> split = largeObject.split(size);
+        largeObject = split.first;
+        nextLargeObject = split.second;
+    }
+    
+    largeObject.setFree(false);
+
+    if (prevLargeObject) {
+        LargeObject merged = prevLargeObject.merge();
+        m_largeObjects.insert(merged);
+    }
+
+    if (nextLargeObject) {
+        LargeObject merged = nextLargeObject.merge();
+        m_largeObjects.insert(merged);
+    }
+
+    return largeObject.begin();
 }
 
 void Heap::deallocateLarge(std::lock_guard<StaticMutex>&, const LargeObject& largeObject)

Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.h (196921 => 196922)


--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.h	2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/Heap.h	2016-02-22 10:19:05 UTC (rev 196922)
@@ -83,7 +83,6 @@
     void deallocateSmallLine(std::lock_guard<StaticMutex>&, SmallLine*);
     void deallocateMediumLine(std::lock_guard<StaticMutex>&, MediumLine*);
 
-    void* allocateLarge(std::lock_guard<StaticMutex>&, LargeObject&, size_t);
     void deallocateLarge(std::lock_guard<StaticMutex>&, const LargeObject&);
 
     void splitLarge(BeginTag*, size_t, EndTag*&, Range&);

Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/LargeObject.h (196921 => 196922)


--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/LargeObject.h	2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/LargeObject.h	2016-02-22 10:19:05 UTC (rev 196922)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2015 Apple Inc. All rights reserved.
+ * Copyright (C) 2015-2016 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -53,7 +53,10 @@
 
     void setFree(bool) const;
     bool isFree() const;
-    
+
+    bool prevCanMerge() const;
+    bool nextCanMerge() const;
+
     Owner owner() const;
     void setOwner(Owner) const;
     
@@ -118,6 +121,20 @@
     return m_beginTag->isFree();
 }
 
+inline bool LargeObject::prevCanMerge() const
+{
+    EndTag* prev = m_beginTag->prev();
+
+    return prev->isFree() && prev->owner() == this->owner();
+}
+
+inline bool LargeObject::nextCanMerge() const
+{
+    BeginTag* next = m_endTag->next();
+
+    return next->isFree() && next->owner() == this->owner();
+}
+
 inline Owner LargeObject::owner() const
 {
     validate();

Modified: releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/VMHeap.h (196921 => 196922)


--- releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/VMHeap.h	2016-02-22 10:12:04 UTC (rev 196921)
+++ releases/WebKitGTK/webkit-2.10/Source/bmalloc/bmalloc/VMHeap.h	2016-02-22 10:19:05 UTC (rev 196922)
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2014, 2015 Apple Inc. All rights reserved.
+ * Copyright (C) 2014-2016 Apple Inc. All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions
@@ -57,7 +57,7 @@
 
     void deallocateSmallPage(std::unique_lock<StaticMutex>&, SmallPage*);
     void deallocateMediumPage(std::unique_lock<StaticMutex>&, MediumPage*);
-    void deallocateLargeObject(std::unique_lock<StaticMutex>&, LargeObject&);
+    void deallocateLargeObject(std::unique_lock<StaticMutex>&, LargeObject);
 
 private:
     LargeObject allocateLargeObject(LargeObject&, size_t);
@@ -95,14 +95,23 @@
 {
     BASSERT(largeObject.isFree());
 
+    LargeObject nextLargeObject;
+
     if (largeObject.size() - size > largeMin) {
         std::pair<LargeObject, LargeObject> split = largeObject.split(size);
         largeObject = split.first;
-        m_largeObjects.insert(split.second);
+        nextLargeObject = split.second;
     }
 
     vmAllocatePhysicalPagesSloppy(largeObject.begin(), largeObject.size());
     largeObject.setOwner(Owner::Heap);
+
+    // Be sure to set the owner for the object we return before inserting the leftover back
+    // into the free list. The free list asserts that we never insert an object that could
+    // have merged with its neighbor.
+    if (nextLargeObject)
+        m_largeObjects.insert(nextLargeObject);
+
     return largeObject.begin();
 }
 
@@ -151,25 +160,29 @@
     m_mediumPages.push(page);
 }
 
-inline void VMHeap::deallocateLargeObject(std::unique_lock<StaticMutex>& lock, LargeObject& largeObject)
+inline void VMHeap::deallocateLargeObject(std::unique_lock<StaticMutex>& lock, LargeObject largeObject)
 {
     largeObject.setOwner(Owner::VMHeap);
-    
-    // If we couldn't merge with our neighbors before because they were in the
-    // VM heap, we can merge with them now.
-    LargeObject merged = largeObject.merge();
 
-    // Temporarily mark this object as allocated to prevent clients from merging
-    // with it or allocating it while we're messing with its physical pages.
-    merged.setFree(false);
+    // Multiple threads might scavenge concurrently, meaning that new merging opportunities
+    // become visible after we reacquire the lock. Therefore we loop.
+    do {
+        // If we couldn't merge with our neighbors before because they were in the
+        // VM heap, we can merge with them now.
+        largeObject = largeObject.merge();
 
-    lock.unlock();
-    vmDeallocatePhysicalPagesSloppy(merged.begin(), merged.size());
-    lock.lock();
+        // Temporarily mark this object as allocated to prevent clients from merging
+        // with it or allocating it while we're messing with its physical pages.
+        largeObject.setFree(false);
 
-    merged.setFree(true);
+        lock.unlock();
+        vmDeallocatePhysicalPagesSloppy(largeObject.begin(), largeObject.size());
+        lock.lock();
 
-    m_largeObjects.insert(merged);
+        largeObject.setFree(true);
+    } while (largeObject.prevCanMerge() || largeObject.nextCanMerge());
+
+    m_largeObjects.insert(largeObject);
 }
 
 } // namespace bmalloc
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to