Hi all:
I've done a benchmark for the asynchronous version of the c++ server,
but the result is terribly poor compared to either the official one
<https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5685265389584384>
or
some other results like this <https://github.com/grpc/grpc/issues/6504> done
by other users, I'm wondering what is wrong with my test, could someone do
me a favor?
There are the details for my test.:
Enrivonment :
- win 10pro 64bit : intel i7 4.0GHZ * 4 physical core * 2 logic core
RAM:16GB
- mac high serria 10.13.6: intel i7 2.2GHZ * 4 physical core * 2 logic
core RAM:16G
*Note: I don't have any linux machines, so there are no results for
it.*
Grpc version:
- win : 77ec6c94ad58bedfdf8f28682914e9893e318be0, around 2018.1
- mac: e3f37b7f4320f0e016a806796e2adaed03bf23f7, around 2019.2
Compiler:
- win: vs2015 Microsoft (R) C/C++ Optimizing Compiler Version
19.00.24215.1 for x86
- mac: Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Parameters:
- number of CQs an server instance has.
- number of threads totally exist, evenly distributed on each CQ.
- number of pool size(which is the *CallData* instances pre-allocated
for each CQ.
Client & Server Behavior:
- Client: Sending arond 30-50k requests to the async server in parallel,
and waiting for all of them to be responded.
- Server: Based on the *greeter_async_server.cc of the helloworld
example* - just do some basically ignorable logic and response
immediately after that.
* The code are in the attachments.*
The result (requests the server can deal per second in average) is :
OS pool 1 thread 2 thread 4 thread 8 thread 16 thread
1 CQ win 100 3647 4016 3366 3012 3427
200 3963 5202 3308 3411 4507
400 3787 4494 3476 3203 3123
mac 100 38880 35803 22629 22128 22321
200 37778 35285 24801 21805 22172
400 39200 36231 23446 22311 22553
2 CQ win 100 3086 4144 4575 3662 3959
200 3334 4354 3588 3507 3536
400 3069 4299 3565 3830 3852
mac 100 39154 32362 31948 25227 23441
200 37792 33046 31908 25169 24142
400 40584 33909 32446 25284 24943
4 CQ win 100 3204 4199 4644 3988 3742
200 3125 4097 3954 3997 3638
400 3207 4196 3731 4008 3526
mac 100 39793 33101 28851 32310 25214
200 38804 31545 32605 32268 24869
400 38819 32776 31786 32133 25207
[image: image.png]
Recap of the result:
- number of CQs basically has nothing to do with the throughput, *which
is unbelievable, in my understanding, it is an critical factor after
learned how the polling-engine
<https://github.com/grpc/grpc/blob/master/doc/core/epoll-polling-engine.md>
works.*
- number of pool size has nothing to do with the throughput,* I'm not
sure whether this is normal or not.*
- number of threads which are evenly distributed among the CQs *has
nothing to do with the throughput on windows, but decreasing it on mac.
This is somewhere wired.*
Besides, I tested the synchronous version(*greeter_server.cc*) of c++
server, giving me the result of 18807 on mac and 3041 on win, which is also
very poor.
I've read the example of *grpc\test\cpp\qps\server_async.cc *which said to
be an good example of how to write a high throughput async c++ server. I
found its approach of enhancing the throughput is more of less the same
with mine:
- scale #CQ.
- scale #thread.
I haven't test the *qps example* on my machine, I haven't find a easier way
to build it and hard to believe to it can achieve a better result with the
same environment of mine.
I know there are something must be wrong, but where are they?
- Thanks a lot.
- Arthur.
--
You received this message because you are subscribed to the Google Groups
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/grpc-io/CAKymdqMw3nC1s7mb2jgQtXX8eGg%2BGZyUz%3DVi07184vwVo44V4Q%40mail.gmail.com.
/*
*
* Copyright 2015 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
#include <memory>
#include <iostream>
#include <string>
#include <thread>
#include <vector>
#include <grpc++/grpc++.h>
#include <grpc/support/log.h>
#include "helloworld.grpc.pb.h"
using grpc::Server;
using grpc::ServerAsyncResponseWriter;
using grpc::ServerBuilder;
using grpc::ServerContext;
using grpc::ServerCompletionQueue;
using grpc::Status;
using helloworld::HelloRequest;
using helloworld::HelloReply;
using helloworld::Greeter;
int g_thread_num = 1;
int g_cq_num = 1;
int g_pool = 1;
class ServerImpl final {
public:
~ServerImpl() {
server_->Shutdown();
// Always shutdown the completion queue after the server.
for (const auto& _cq : m_cq)
_cq->Shutdown();
}
// There is no shutdown handling in this code.
void Run() {
std::string server_address("0.0.0.0:50051");
ServerBuilder builder;
// Listen on the given address without any authentication mechanism.
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
// Register "service_" as the instance through which we'll communicate with
// clients. In this case it corresponds to an *asynchronous* service.
builder.RegisterService(&service_);
// Get hold of the completion queue used for the asynchronous communication
// with the gRPC runtime.
for (int i = 0; i < g_cq_num; ++i) {
//cq_ = builder.AddCompletionQueue();
m_cq.emplace_back(builder.AddCompletionQueue());
}
// Finally assemble the server.
server_ = builder.BuildAndStart();
std::cout << "Server listening on " << server_address << std::endl;
// Proceed to the server's main loop.
std::vector<std::thread*> _vec_threads;
for (int i = 0; i < g_thread_num; ++i) {
int _cq_idx = i % g_cq_num;
for (int j = 0; j < g_pool; ++j)
new CallData(&service_, m_cq[_cq_idx].get());
_vec_threads.emplace_back(new std::thread(&ServerImpl::HandleRpcs,
this, _cq_idx));
}
std::cout << g_thread_num << " working aysnc threads spawned" << std::endl;
for (const auto& _t : _vec_threads)
_t->join();
}
private:
// Class encompasing the state and logic needed to serve a request.
class CallData {
public:
// Take in the "service" instance (in this case representing an asynchronous
// server) and the completion queue "cq" used for asynchronous communication
// with the gRPC runtime.
CallData(Greeter::AsyncService* service, ServerCompletionQueue* cq)
: service_(service), cq_(cq), responder_(&ctx_), status_(CREATE) {
// Invoke the serving logic right away.
Proceed();
}
void Proceed() {
if (status_ == CREATE) {
// Make this instance progress to the PROCESS state.
status_ = PROCESS;
// As part of the initial CREATE state, we *request* that the system
// start processing SayHello requests. In this request, "this" acts are
// the tag uniquely identifying the request (so that different CallData
// instances can serve different requests concurrently), in this case
// the memory address of this CallData instance.
service_->RequestSayHello(&ctx_, &request_, &responder_, cq_, cq_,
this);
} else if (status_ == PROCESS) {
// Spawn a new CallData instance to serve new clients while we process
// the one for this CallData. The instance will deallocate itself as
// part of its FINISH state.
new CallData(service_, cq_);
// The actual processing.
std::string prefix("Hello ");
reply_.set_message(prefix + request_.name());
// And we are done! Let the gRPC runtime know we've finished, using the
// memory address of this instance as the uniquely identifying tag for
// the event.
status_ = FINISH;
responder_.Finish(reply_, Status::OK, this);
} else {
GPR_ASSERT(status_ == FINISH);
// Once in the FINISH state, deallocate ourselves (CallData).
delete this;
}
}
private:
// The means of communication with the gRPC runtime for an asynchronous
// server.
Greeter::AsyncService* service_;
// The producer-consumer queue where for asynchronous server notifications.
ServerCompletionQueue* cq_;
// Context for the rpc, allowing to tweak aspects of it such as the use
// of compression, authentication, as well as to send metadata back to the
// client.
ServerContext ctx_;
// What we get from the client.
HelloRequest request_;
// What we send back to the client.
HelloReply reply_;
// The means to get back to the client.
ServerAsyncResponseWriter<HelloReply> responder_;
// Let's implement a tiny state machine with the following states.
enum CallStatus { CREATE, PROCESS, FINISH };
CallStatus status_; // The current serving state.
};
// This can be run in multiple threads if needed.
void HandleRpcs(int cq_idx) {
uint32_t _counter = 0;
// Spawn a new CallData instance to serve new clients.
void* tag; // uniquely identifies a request.
bool ok;
while (true) {
// Block waiting to read the next event from the completion queue. The
// event is uniquely identified by its tag, which in this case is the
// memory address of a CallData instance.
// The return value of Next should always be checked. This return value
// tells us whether there is any kind of event or cq_ is shutting down.
//GPR_ASSERT(cq_->Next(&tag, &ok));
GPR_ASSERT(m_cq[cq_idx]->Next(&tag, &ok));
GPR_ASSERT(ok);
//std::cout << "thread " << std::this_thread::get_id() << " receive " <<
++_counter << " reqs." << std::endl;
static_cast<CallData*>(tag)->Proceed();
}
}
//std::unique_ptr<ServerCompletionQueue> cq_;
std::vector<std::unique_ptr<ServerCompletionQueue>> m_cq;
Greeter::AsyncService service_;
std::unique_ptr<Server> server_;
};
const char* ParseCmdPara( char* argv,const char* para) {
auto p_target = std::strstr(argv,para);
if (p_target == nullptr) {
printf("para error argv[%s] should be %s \n",argv,para);
return nullptr;
}
p_target += std::strlen(para);
return p_target;
}
int main(int argc, char** argv) {
if (argc != 4) {
std::cout << "Usage:./program --thread=xx --cq=xx --pool=xx";
return 0;
}
g_thread_num = std::atoi(ParseCmdPara(argv[1],"--thread="));
g_cq_num = std::atoi(ParseCmdPara(argv[2],"--cq="));
g_pool = std::atoi(ParseCmdPara(argv[3],"--pool="));
ServerImpl server;
server.Run();
return 0;
}
/*
*
* Copyright 2015 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
#include <iostream>
#include <memory>
#include <string>
#include <grpc++/grpc++.h>
#include <grpc/support/log.h>
#include <thread>
#include "helloworld.grpc.pb.h"
using grpc::Channel;
using grpc::ClientAsyncResponseReader;
using grpc::ClientContext;
using grpc::CompletionQueue;
using grpc::Status;
using helloworld::HelloRequest;
using helloworld::HelloReply;
using helloworld::Greeter;
class GreeterClient {
public:
explicit GreeterClient(std::shared_ptr<Channel> channel,uint32_t count)
: stub_(Greeter::NewStub(channel)),m_total(count) {}
// Assembles the client's payload and sends it to the server.
void SayHello(const std::string& user) {
// Data we are sending to the server.
HelloRequest request;
request.set_name(user);
// Call object to store rpc data
AsyncClientCall* call = new AsyncClientCall;
// stub_->PrepareAsyncSayHello() creates an RPC object, returning
// an instance to store in "call" but does not actually start the RPC
// Because we are using the asynchronous API, we need to hold on to
// the "call" instance in order to get updates on the ongoing RPC.
call->response_reader =
stub_->PrepareAsyncSayHello(&call->context, request, &cq_);
// StartCall initiates the RPC call
call->response_reader->StartCall();
// Request that, upon completion of the RPC, "reply" be updated with the
// server's response; "status" with the indication of whether the
operation
// was successful. Tag the request with the memory address of the call
object.
call->response_reader->Finish(&call->reply, &call->status, (void*)call);
}
// Loop while listening for completed responses.
// Prints out the response from the server.
void AsyncCompleteRpc() {
void* got_tag;
bool ok = false;
uint32_t _counter = 0;
auto _start = std::chrono::steady_clock::now();
// Block until the next result is available in the completion queue
"cq".
while (cq_.Next(&got_tag, &ok)) {
// The tag in this example is the memory location of the call object
AsyncClientCall* call = static_cast<AsyncClientCall*>(got_tag);
// Verify that the request was completed successfully. Note that
"ok"
// corresponds solely to the request for updates introduced by
Finish().
GPR_ASSERT(ok);
GPR_ASSERT(call->status.ok());
/*
if (call->status.ok())
std::cout << "Greeter received: " << call->reply.message() <<
std::endl;
else
std::cout << "RPC failed" << std::endl;
*/
// Once we're complete, deallocate the call object.
delete call;
if (++_counter >= m_total)
break;
}
std::cout << "m_total:" << m_total << std::endl;
auto _end = std::chrono::steady_clock::now();
auto _ms = std::chrono::duration_cast<std::chrono::milliseconds>(_end -
_start);
std::cout << "time cost:" << _ms.count() << std::endl;
uint32_t _throughput = m_total / float(_ms.count()) * 1000;
std::cout << "throughput : " << _throughput << std::endl;
}
private:
// struct for keeping state and data information
struct AsyncClientCall {
// Container for the data we expect from the server.
HelloReply reply;
// Context for the client. It could be used to convey extra information
to
// the server and/or tweak certain RPC behaviors.
ClientContext context;
// Storage for the status of the RPC upon completion.
Status status;
std::unique_ptr<ClientAsyncResponseReader<HelloReply>> response_reader;
};
// Out of the passed in Channel comes the stub, stored here, our view of the
// server's exposed services.
std::unique_ptr<Greeter::Stub> stub_;
// The producer-consumer queue we use to communicate asynchronously with the
// gRPC runtime.
CompletionQueue cq_;
uint32_t m_total;
};
int main(int argc, char** argv) {
uint32_t count = 500000;
if (argc != 2) {
std::cout << "Usage:./program --count=xx";
return 0;
}
const char * target_str = "--count=";
auto p_target = std::strstr(argv[1],target_str);
if (p_target == nullptr) {
printf("para error argv[1] should be --count=xx \n");
return 0;
}
p_target += std::strlen(target_str);
count = std::atoi(p_target);
std::cout << "Total req:" << count << std::endl;
// Instantiate the client. It requires a channel, out of which the actual
RPCs
// are created. This channel models a connection to an endpoint (in this
case,
// localhost at port 50051). We indicate that the channel isn't
authenticated
// (use of InsecureChannelCredentials()).
GreeterClient greeter(grpc::CreateChannel(
"localhost:50051", grpc::InsecureChannelCredentials()),count);
for (uint32_t i = 0; i < count; i++) {
std::string user("world " + std::to_string(i));
greeter.SayHello(user); // The actual RPC call!
}
std::this_thread::sleep_for(std::chrono::milliseconds(30));
// Spawn reader thread that loops indefinitely
std::thread thread_ = std::thread(&GreeterClient::AsyncCompleteRpc,
&greeter);
//std::cout << "Press control-c to quit" << std::endl << std::endl;
thread_.join(); //blocks forever
return 0;
}
/*
*
* Copyright 2015 gRPC authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
#include <iostream>
#include <memory>
#include <string>
#include <chrono>
#include <grpc++/grpc++.h>
#ifdef BAZEL_BUILD
#include "examples/protos/helloworld.grpc.pb.h"
#else
#include "helloworld.grpc.pb.h"
#endif
using grpc::Server;
using grpc::ServerBuilder;
using grpc::ServerContext;
using grpc::Status;
using helloworld::HelloRequest;
using helloworld::HelloReply;
using helloworld::Greeter;
int g_sleep = 3;
// Logic and data behind the server's behavior.
class GreeterServiceImpl final : public Greeter::Service {
Status SayHello(ServerContext* context, const HelloRequest* request,
HelloReply* reply) override {
//std::cout << "client peer addr:" << context->peer() << std::endl;
//std::cout << "receive.." << request->DebugString() << std::endl;
std::string prefix("Hello ");
reply->set_message(prefix + request->name());
/*
std::chrono::seconds _sec(g_sleep);
std::cout << "begin sleep.." << std::endl;
std::this_thread::sleep_for(_sec);
std::cout << "end sleep.." << std::endl;
*/
return Status::OK;
}
};
void RunServer(int port) {
#define ADDR_BUFF_SIZE (50)
char sz_addr[ADDR_BUFF_SIZE] = { 0 };
std::memset(sz_addr, 0, ADDR_BUFF_SIZE);
std::snprintf(sz_addr,ADDR_BUFF_SIZE,"0.0.0.0:%d",port);
std::string server_address(sz_addr);
GreeterServiceImpl service;
ServerBuilder builder;
// Listen on the given address without any authentication mechanism.
builder.AddListeningPort(server_address, grpc::InsecureServerCredentials());
// Register "service" as the instance through which we'll communicate with
// clients. In this case it corresponds to an *synchronous* service.
builder.RegisterService(&service);
// Finally assemble the server.
std::unique_ptr<Server> server(builder.BuildAndStart());
std::cout << "Server listening on " << server_address << std::endl;
// Wait for the server to shutdown. Note that some other thread must be
// responsible for shutting down the server for this call to ever return.
server->Wait();
}
int main(int argc, char** argv) {
int port = 50051;
if (argc <= 1) {
std::cout << "Usage:./program --port=xx --sleep=xx";
return 0;
}
const char * target_str = "--port=";
auto p_target = std::strstr(argv[1],target_str);
if (p_target == nullptr) {
printf("para error argv[1] should be --port=xx...\n");
return 0;
}
p_target += std::strlen(target_str);
port = std::atoi(p_target);
if (argc == 3) {
target_str = "--sleep=";
p_target = std::strstr(argv[2],target_str);
if (p_target != nullptr) {
p_target += std::strlen(target_str);
g_sleep = std::atoi(p_target);
}
}
RunServer(port);
return 0;
}
