Hi,
Den 12. aug. 2011 14:30, skrev ext [email protected]:
Hi all!
There are 3 tests detailed in this mail:
1. ExternalResource garbage collector performance
2. Isolate/Context performance comparisons
3. Isolate/Context memory usage comparisons
1. ExternalResource garbage collector performance:
I messed up the ExternalResource test I had sent to Jędrek before
so here is a new test case (external-resource-tester.cpp). I used a
helper class for measuring which I also attached to this mail
(tester.h, tester.cpp). A sample run with 100 test cases:
Mean of ExternalResource : 22.34 ms.
Deviation of ExternalResource : 0.19 ms.
Number of Point objects : 0.
Mean of MakeWeak : 33.82 ms.
Deviation of MakeWeak : 0.32 ms.
Number of Point objects : 0.
Thanks for the tests!
MeasureExternalResource() creates persistent handles. Is there a reason
for that? You could just use local handles. The key advantage of using
external resources is that we shouldn't have to create persistent
handles at all if the object is owned by JS.
MeasureExternalResource() and MeasureMakeWeak() only measure the time of
the "final" garbage collections, after all objects have been created.
But what about the effects of implicit GC calls as more objects are
allocated?
I've attached a modified version of external-resource-tester.cpp to
explore this. Noticeable changes are:
- Creates the objects on the JS side for added "realism"
- Uses V8::AdjustAmountOfExternalAllocatedMemory() to inform V8 about
the size of external memory we (de)allocate
- Initializes V8's flags from a V8ARGS environment variable, so that we
can e.g. enable tracing of GC calls
- Prints the number of Point objects that are still around _before_
calling V8::IdleNotification()
Now, if I run V8ARGS="--trace_gc" ./external-resource-tester, I get:
*** Begin MeasureExternalResource ***
Scavenge 0.9 -> 0.5 MB, 2 ms.
Scavenge 0.9 -> 0.5 MB, 3 ms.
Scavenge 1.0 -> 0.5 MB, 3 ms.
[25 more like this ...]
Scavenge 1.0 -> 0.5 MB, 3 ms.
Scavenge 1.0 -> 0.5 MB, 3 ms.
Number of Point objects before IdleNotification: _31416_
Scavenge 0.9 -> 0.5 MB, 3 ms.
Mark-sweep 0.5 -> 0.4 MB, 0 ms.
Mark-compact 0.4 -> 0.4 MB, 1 ms.
*** End MeasureExternalResource ***
*** Begin MeasureMakeWeak ***
Scavenge 0.9 -> 0.9 MB, 1 ms.
Scavenge 1.3 -> 1.3 MB, 1 ms.
Scavenge 2.0 -> 2.0 MB, 3 ms.
Mark-sweep 2.8 -> 2.8 MB, 15 / 24 ms.
Scavenge 4.8 -> 4.8 MB, 6 ms.
Scavenge 6.3 -> 6.3 MB, 6 ms.
Scavenge 9.3 -> 9.3 MB, 14 ms.
Mark-sweep 12.3 -> 9.9 MB, 62 / 101 ms.
Number of Point objects before IdleNotification: _222325_
Scavenge 13.3 -> 13.3 MB, 11 ms.
Mark-sweep 13.3 -> 3.8 MB, 22 / 40 ms.
Mark-compact 3.8 -> 0.4 MB, 4 ms.
*** End MeasureMakeWeak ***
Mean of ExternalResource : 5.66 ms.
Deviation of ExternalResource : 0.00 ms.
Number of Point objects : 0.
Mean of MakeWeak : 57.62 ms.
Deviation of MakeWeak : 0.00 ms.
Number of Point objects : 0.
Note that the scavenger is invoked _a lot_ more often for
ExternalResource, and that pretty much all Point objects have been
collected before we even call IdleNotification(). This also explains why
the measured part of the test is now 10x faster for ExternalResource
compared to MakeWeak; most of the work was done already. Also note how
large the heap grows for MakeWeak. This all seems to support the claim
made in the external resource patch commit message: "using a weak
persistent handle for this purpose [...] seems to incur a fairly massive
performance penalty for short lived objects as weak persistent handle
callbacks are not called until the object has been promoted into the old
object space".
(By the way, there are a lot of other V8 flags that affect GC that you
might want to play with when running the test -- see
v8/src/flag-definitions.h.)
Anyways, there seem to be big gains possible from this patch. I see that
Chromium uses the persistent+MakeWeak() for its JS/DOM bindings as well.
Maybe a way to make the change more compelling/relevant upstream would
be to convert Chromium's bindings to use external resources and
demonstrate performance improvements, e.g. on the Dromeo benchmark.
2. Isolate/Context memory usage comparisons:
We used a Valgrind tool called Freya (see
http://webkit.sed.hu/node/29) to measure the memory consumption of the
test cases.
The test cases use greatly simplified Engine-like classes as follows:
- Using an own Context and the default Isolate (context-memtest.cpp)
- Using an own Context and an own Isolate (isolate-memtest.cpp)
- Using an own Isolate and no Context at all (pure-isolate-memtest.cpp)
We tested each cases with 1, 5, 10 and 15 instances of test objects.
The results are:
- context-memtest 1: 984.0Kb, 5: 1.7Mb, 10: 2.5Mb, 15: 2.9Mb
- isolate-memtest 1: 1.4Mb, 5: 4.1Mb, 10: 7.6Mb, 15: 11.1Mb
- pure-isolate-memtest 1: 796.0Kb, 5: 788.0Kb, 10: 792.0Kb, 15: 792.0Kb
It would be good if these tests did a simple evaluation (e.g.
"Math.abs(-1)") in each "engine", just to make sure that the whole JS
environment has been initialized (in case some of the Isolate/Context
initialization is done lazily).
Also, the pure-isolate-memtest results don't make much sense to me.
3. Isolate/Context performance comparisons:
These tests are also use simplified Engine-like classes as the
ones in the memory benchmarks and the same Tester class as the
ExternalResource test (isolate-context-tester.cpp, tester.h,
tester.cpp). A sample run with 100 test cases:
Mean:
Own Isolate : 34.86 ms
Default Isolate : 5.47 ms
Pure Isolate : 30.26 ms
Deviation:
Own Isolate : 0.21 ms
Default Isolate : 0.02 ms
Pure Isolate : 0.18 ms
OK, so entering/exiting isolates seems to be a lot more expensive than
entering/exiting just contexts.
What we'd like to know is how expensive it is compared to doing "useful
work" in V8.
For example, given code like this:
QJSEngine eng;
QJSValue array = eng.newArray();
for (int i = 0; i < 1000; ++i) {
QJSValue person = eng.newObject();
person.setProperty("name", "John Doe");
person.setProperty("age", 19);
person.setProperty("food", "bacon");
array.setProperty(i, person);
}
what is the ratio of the isolate/context bookkeeping compared to "useful
work" (having the array populated with objects), for each of the schemes
(could also do a "pure V8" API version for reference). There is even a
fourth scheme we were considering: Having a QJSEngine::makeCurrent()
function that enters the associated isolate/context, and from then on
all API calls will implicitly use it (i.e. no calls to
Isolate/Context::Enter/Exit in newObject() and friends). This is the
closest to raw V8 performance we can get. From a Qt API philosophy, we'd
like to avoid this design, since it's error-prone. But if the cost of
per-API-call entry/exit is too high, we might have to do it.
I suggest adding more tests that do useful work in the doStuff()
functions. From simple stuff like Value::ToNumber(), to more complex
things like evaluating scripts, and creating objects.
Best regards,
Kent
/*
* Benchmark - ExternalResource
* Usage: external-resource-tester [number_of_tests]
*
*/
#include <v8.h>
#include <cassert>
#include <cstdlib>
#include <cstring>
#include "tester.h"
using namespace v8;
using namespace std;
void ExecuteString(v8::Handle<v8::String> source, bool print_result)
{
v8::HandleScope handle_scope;
v8::TryCatch try_catch;
v8::Handle<v8::Script> script = v8::Script::Compile(source);
assert(!script.IsEmpty());
v8::Handle<v8::Value> result = script->Run();
assert(!result.IsEmpty());
if (print_result) {
v8::String::Utf8Value str(result);
printf("%s\n", *str);
}
}
class Point {
public:
static int counter;
Point(int x, int y) : x_(x), y_(y)
{
++counter;
}
~Point()
{
--counter;
}
int x_;
int y_;
};
int Point::counter = 0;
class V8ObjectResource : public v8::Object::ExternalResource {
public:
V8ObjectResource() {}
enum ResourceType { POINT_TYPE, SOME_OTHER_TYPE };
virtual ResourceType getResourceType() const = 0;
};
class V8PointResource : public V8ObjectResource {
public:
V8PointResource(Point* p) : point(p) {}
~V8PointResource()
{
delete point;
V8::AdjustAmountOfExternalAllocatedMemory(-int(sizeof(V8PointResource) + sizeof(Point)));
}
enum { V8_RESOURCE_TYPE = V8ObjectResource::POINT_TYPE };
V8ObjectResource::ResourceType getResourceType() const
{
return V8ObjectResource::POINT_TYPE;
}
//private:
Point* point;
};
template <class T>
inline T* v8_resource_cast(Handle<Object> object)
{
V8ObjectResource* resource = static_cast<V8ObjectResource*>(object->GetExternalResource());
return (resource &&
(uint32_t)resource->getResourceType() == (uint32_t)T::V8_RESOURCE_TYPE) ? static_cast<T*>(resource) : 0;
}
static Handle<Value> makeResourcePoint(const Arguments& args)
{
assert(args.IsConstructCall());
Local<Object> self = args.This();
Point* p = new Point(10, 20);
V8PointResource* point_resource = new V8PointResource(p);
V8::AdjustAmountOfExternalAllocatedMemory(int(sizeof(V8PointResource) + sizeof(Point)));
self->SetExternalResource(point_resource);
return v8::Undefined();
}
static Handle<Value> getResourcePointX(Local<String>, const AccessorInfo &info)
{
Local<Object> self = info.Holder();
V8PointResource *res = v8_resource_cast<V8PointResource>(self);
assert(res != 0);
return v8::Int32::New(res->point->x_);
}
static Handle<Value> getResourcePointY(Local<String>, const AccessorInfo &info)
{
Local<Object> self = info.Holder();
V8PointResource *res = v8_resource_cast<V8PointResource>(self);
assert(res != 0);
return v8::Int32::New(res->point->y_);
}
void weakPointCallback(Persistent<Value> object, void* parameter)
{
Point* point = static_cast<Point*>(parameter);
delete point;
object.Dispose();
object.Clear();
V8::AdjustAmountOfExternalAllocatedMemory(-int(sizeof(Point)));
}
static Handle<Value> makeWeakPoint(const Arguments& args)
{
assert(args.IsConstructCall());
Local<Object> self = args.This();
assert(self->InternalFieldCount() == 1);
Point *point = new Point(10, 20);
self->SetPointerInInternalField(0, point);
V8::AdjustAmountOfExternalAllocatedMemory(int(sizeof(Point)));
Persistent<Object> weakSelf = Persistent<Object>::New(self);
weakSelf.MakeWeak(point, weakPointCallback);
return weakSelf;
}
static Handle<Value> getWeakPointX(Local<String>, const AccessorInfo &info)
{
Local<Object> self = info.Holder();
Point *point = static_cast<Point*>(self->GetPointerFromInternalField(0));
return v8::Int32::New(point->x_);
}
static Handle<Value> getWeakPointY(Local<String>, const AccessorInfo &info)
{
Local<Object> self = info.Holder();
Point *point = static_cast<Point*>(self->GetPointerFromInternalField(0));
return v8::Int32::New(point->y_);
}
void MeasureExternalResource(int counter, Tester* tester)
{
printf("*** Begin MeasureExternalResource ***\n");
v8::Local<v8::FunctionTemplate> fun_templ = v8::FunctionTemplate::New(makeResourcePoint);
v8::Handle<v8::ObjectTemplate> point_templ = fun_templ->InstanceTemplate();
point_templ->SetHasExternalResource(true);
point_templ->SetAccessor(v8::String::New("x"), getResourcePointX);
point_templ->SetAccessor(v8::String::New("y"), getResourcePointY);
v8::Context::GetEntered()->Global()->Set(v8::String::New("Point"), fun_templ->GetFunction());
v8::Context::GetEntered()->Global()->Set(v8::String::New("counter"), v8::Int32::New(counter));
ExecuteString(v8::String::New("for (i = 0; i < counter; ++i) { new Point(); }"), false);
printf("Number of Point objects before IdleNotification: %d\n", Point::counter);
tester->startMeasuring();
while (!V8::IdleNotification()) { }
tester->endMeasuring();
assert(Point::counter == 0);
printf("*** End MeasureExternalResource ***\n");
}
void MeasureMakeWeak(int counter, Tester* tester)
{
printf("*** Begin MeasureMakeWeak ***\n");
v8::Local<v8::FunctionTemplate> fun_templ = v8::FunctionTemplate::New(makeWeakPoint);
v8::Handle<v8::ObjectTemplate> point_templ = fun_templ->InstanceTemplate();
point_templ->SetInternalFieldCount(1);
point_templ->SetAccessor(v8::String::New("x"), getWeakPointX);
point_templ->SetAccessor(v8::String::New("y"), getWeakPointY);
v8::Context::GetEntered()->Global()->Set(v8::String::New("Point"), fun_templ->GetFunction());
v8::Context::GetEntered()->Global()->Set(v8::String::New("counter"), v8::Int32::New(counter));
ExecuteString(v8::String::New("for (i = 0; i < counter; ++i) { new Point(); }"), false);
printf("Number of Point objects before IdleNotification: %d\n", Point::counter);
tester->startMeasuring();
while (!V8::IdleNotification()) { }
tester->endMeasuring();
assert(Point::counter == 0);
printf("*** End MeasureMakeWeak ***\n");
}
int main(int argc, char* argv[])
{
if (argc > 2) {
return 666;
}
int number_of_tests = 1;
if (argc == 2) {
int num = atoi(argv[1]);
if (num) {
number_of_tests = num;
} else {
return 666;
}
}
const char *v8args = getenv("V8ARGS");
if (v8args)
v8::V8::SetFlagsFromString(v8args, strlen(v8args));
HandleScope handle_scope;
v8::Handle<v8::ObjectTemplate> global = v8::ObjectTemplate::New();
Persistent<Context> context = Context::New(NULL, global);
Context::Scope context_scope(context);
const int OBJECT_COUNTER = 1000000;
Tester* externalTester = new Tester();
Tester* weakTester = new Tester();
for (int i = 0; i < number_of_tests; ++i) {
MeasureExternalResource(OBJECT_COUNTER, externalTester);
MeasureMakeWeak(OBJECT_COUNTER, weakTester);
}
printf("Mean of ExternalResource : %.2lf ms.\n", externalTester->getMean());
printf("Deviation of ExternalResource : %.2lf ms.\n", externalTester->getDeviation());
printf("Number of Point objects : %d.\n", Point::counter);
printf("Mean of MakeWeak : %.2lf ms.\n", weakTester->getMean());
printf("Deviation of MakeWeak : %.2lf ms.\n", weakTester->getDeviation());
printf("Number of Point objects : %d.\n", Point::counter);
context.Dispose();
return 0;
}
_______________________________________________
Qt-script mailing list
[email protected]
http://lists.qt.nokia.com/mailman/listinfo/qt-script
