date:20180124

Re: [zeromq-dev] Pattern for two clients and one server

2018-01-24 Thread Bob Eby

I recently wrote a client server application that seems a bit similar to yours.

> How should I pair the request/reply calls?

Initially I used single threaded REQ / REP on client and server to test.

In order to avoid the server blocking, I created multiple worker
threads on server each connected back to a ROUTER / DEALER
configuration as in Figure 16 of http://zguide.zeromq.org/page:all

It only took me about 2 days to go from a nearly complete one client
to one server application to fully functional multiple client / many
threaded server.  (I even built the server so if I ran the .exe twice,
the second instance would fire up more threads in a separate process!
All done in the 2 days.)

I did have to be sure that my server code was thread-safe, but given I
was already using an SQL database for most of the heavy lifting this
was very easy.

> I need the communication to be asynchronous, without doing busy waiting.

I'm not entirely sure what you mean here.  It's up to the client to be
sure that it is not waiting around for a server on slow operations.
Also, it is important to note that even for multi-part messages, all
of a full message must be stored up before it can be sent and
processed by the other end.  So breaking apart long slow operations
into more smaller ones is very important.

> I went through the ZMQ guide but still I am not pretty sure whether this is
> the right approach.

I would say start with something very simple and plan to reconfigure
the design later.  If you're not sure that will work, write a hello
client server with REQ / REP and get it fully working with 1 server 1
client, then pick another pattern from the guide and try switching to
it to see how much effort it will (or won't) be.

The biggest limitation of REQ / REP will be that every single
operation will have to start with a client asking for something and
then end with a timeout / error or a valid response from the server.
It's a little reminiscent of "stateless" HTTP...

Good luck,
Robert Eby
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Communication between C++ zmq client and Python zmq Server

2018-01-24 Thread Thomas Rodgers

You can have a look at Python’s ctypes module, which will let you define a
‘struct’ from Python with the same layout as your C++ struct.

You can also investigate any number of serialization libraries that have
C++ and Python support, eg ProtoBufs or Thrift, or MagPack or whatever.

On Wed, Jan 24, 2018 at 5:26 PM Bernardo Augusto García Loaiza <
botib...@gmail.com> wrote:

> Hi, ZMQ people.
> Greetings.
>
>
> I have a  C++ zeromq client process in which I am sending some data
> members structures
>
> *ZMQComponent.h* file
>
>
> #include 
> #include 
>
> // To Quat datatype
> #include 
> using sofa::defaulttype::Quat;
>
> using std::string;
>
> namespace sofa
> {
>
> namespace component
> {
>
> namespace controller
> {
>
> /* data structure which I want send data to python zmq server */
> struct instrumentData
> {
> typedef sofa::defaulttype::Vec3d Vec3d;
> Vec3d pos;
> Quat quat;
> int btnState;
> float openInst;
> bool blnDataReady;
> };
>
> class ZMQComponent : public sofa::core::behavior::BaseController
> {
> public:
> SOFA_CLASS(ZMQComponent, sofa::core::behavior::BaseController);
>
> ZMQComponent();
> virtual ~ZMQComponent();
> /* Conect to ZMQ external python Server */
> void setupConnection();
>
> /* Send some data memeber instrumentData structure to ZMQ external Server
> */
> void instrumentDataSend(instrumentData a);
>
> /* initialize function */
> void init();
>
> };
>
> } // namespace sofa
>
> } // namespace component
>
> } // namespace controller
>
>
> The *ZMQComponent.cpp* is:
>
> #include 
> #include 
> #include 
> #include 
> #include "ZMQComponent.h"
>
>
> using namespace std;
>
> namespace sofa
> {
>
> namespace component
> {
>
> namespace controller
> {
>
> /* ZMQ Internal Client context and socket */
> zmq::context_t context(1);
> zmq::socket_t socket(context, ZMQ_REQ);
>
> ZMQComponent::ZMQComponent(){}
>
> void ZMQComponent::setupConnection()
> {
> cout << "Connecting to python zeroMQ server ..." << endl;
> socket.connect("tcp://localhost:");
> }
>
> void ZMQComponent::instrumentDataSend(instrumentData a)
> {
> /* Initialize the data members structure instrumentData */
> a.pos = sofa::defaulttype::Vec3d(1.0f, 1.0f, 1.0f);
> a.quat = defaulttype::Quat(1.0f, 1.0f, 4.0f, 1.0f);
> a.btnState = 5671;
> a.openInst = 1.0f;
> a.blnDataReady = false;
>
> /* We send the btnState data */
> zmq::message_t request(10);
> cout << "The data are: " << a.btnState;
>
> /* We ask for the memory address to ge the btnState content and send it. */
> memcpy(request.data(), , 10);
> socket.send(request);
> }
>
>
> /* In the init function we create the objects to setup connection and send
> data */
> void ZMQComponent::init()
> {
> std::cout << "ZeroMQCommunication::init()" << std::endl;
> ZMQComponent z;
> z.setupConnection();
>
> instrumentData itemp;
> z.instrumentDataSend(itemp);
> }
>
> /* Other code related  */
> ZMQComponent::~ZMQComponent(){}
>
> // int ZeroMqComponentClass = sofa::core::RegisterObject("This component
> does nothing.").add();
> SOFA_DECL_CLASS(ZMQServerComponent)
>
> int ZMQServerComponentClass = sofa::core::RegisterObject("This component
> create a Socket.").add< ZMQServerComponent >();
> } // namespace controller
>
> } // namespace component
>
> } // namespace sofa
>
> Then , my python zmq server  which receive the *btnState*  int variable
> is:
>
> import time
> import zmq
>
> context = zmq.Context()
> socket = context.socket(zmq.REP)
> socket.bind("tcp://*:")
> print('ZMQ Server listening ... ')
>
> while True:
> # Wait for next request from client
> message = socket.recv()
> print("Received message from Sofa: {}".format(message))
> # print("Received message from c++ %s" % message)
>
> # Do some 'work'
> time.sleep(1)
>
> # Send reply back to client
> # socket.send(b"Hola cliente, muy bien y tu ?")
> # print('Response sent')
>
>
>
> The output or the message which arrive to python zmq server is the symbols
> characters of the btnState:
>
> (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python server.py
> ZMQ Server listening ...
> *Received message from Sofa: b"\x00'\x84)\xff\x7f\x00\x00\x98&*"
>
> Likely, As I in my python server I am representing like  a string message
> arrived and I am sending from my c++ client a int btnState data,
> then I try convert int to string in  the python server side:
>
> And I replace this line
>
> print("Received message from Sofa: {}".format(message))
>
> by thiis line
>
> print("Received message from c++ %s" % str(message))
>
> But my output is:
>
> (cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python server.py
> ZMQ Server listening ...
> *Received message from c++ b'\x00^n&\xff\x7f\x00\x00\xd8\\'*
> Traceback (most recent call last):
>   File "server.py", line 19, in 
> message = socket.recv()
>   File "zmq/backend/cython/socket.pyx", line 693, in
> zmq.backend.cython.socket.Socket.recv
>   File "zmq/backend/cython/socket.pyx", line 727, in
> zmq.backend.cython.socket.Socket.recv
>   File "zmq/backend/cython/socket.pyx",

[zeromq-dev] Communication between C++ zmq client and Python zmq Server

2018-01-24 Thread Bernardo Augusto García Loaiza

Hi, ZMQ people.
Greetings.


I have a  C++ zeromq client process in which I am sending some data members
structures

*ZMQComponent.h* file


#include 
#include 

// To Quat datatype
#include 
using sofa::defaulttype::Quat;

using std::string;

namespace sofa
{

namespace component
{

namespace controller
{

/* data structure which I want send data to python zmq server */
struct instrumentData
{
typedef sofa::defaulttype::Vec3d Vec3d;
Vec3d pos;
Quat quat;
int btnState;
float openInst;
bool blnDataReady;
};

class ZMQComponent : public sofa::core::behavior::BaseController
{
public:
SOFA_CLASS(ZMQComponent, sofa::core::behavior::BaseController);

ZMQComponent();
virtual ~ZMQComponent();
/* Conect to ZMQ external python Server */
void setupConnection();

/* Send some data memeber instrumentData structure to ZMQ external Server */
void instrumentDataSend(instrumentData a);

/* initialize function */
void init();

};

} // namespace sofa

} // namespace component

} // namespace controller


The *ZMQComponent.cpp* is:

#include 
#include 
#include 
#include 
#include "ZMQComponent.h"


using namespace std;

namespace sofa
{

namespace component
{

namespace controller
{

/* ZMQ Internal Client context and socket */
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_REQ);

ZMQComponent::ZMQComponent(){}

void ZMQComponent::setupConnection()
{
cout << "Connecting to python zeroMQ server ..." << endl;
socket.connect("tcp://localhost:");
}

void ZMQComponent::instrumentDataSend(instrumentData a)
{
/* Initialize the data members structure instrumentData */
a.pos = sofa::defaulttype::Vec3d(1.0f, 1.0f, 1.0f);
a.quat = defaulttype::Quat(1.0f, 1.0f, 4.0f, 1.0f);
a.btnState = 5671;
a.openInst = 1.0f;
a.blnDataReady = false;

/* We send the btnState data */
zmq::message_t request(10);
cout << "The data are: " << a.btnState;

/* We ask for the memory address to ge the btnState content and send it. */
memcpy(request.data(), , 10);
socket.send(request);
}


/* In the init function we create the objects to setup connection and send
data */
void ZMQComponent::init()
{
std::cout << "ZeroMQCommunication::init()" << std::endl;
ZMQComponent z;
z.setupConnection();

instrumentData itemp;
z.instrumentDataSend(itemp);
}

/* Other code related  */
ZMQComponent::~ZMQComponent(){}

// int ZeroMqComponentClass = sofa::core::RegisterObject("This component
does nothing.").add();
SOFA_DECL_CLASS(ZMQServerComponent)

int ZMQServerComponentClass = sofa::core::RegisterObject("This component
create a Socket.").add< ZMQServerComponent >();
} // namespace controller

} // namespace component

} // namespace sofa

Then , my python zmq server  which receive the *btnState*  int variable is:

import time
import zmq

context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:")
print('ZMQ Server listening ... ')

while True:
# Wait for next request from client
message = socket.recv()
print("Received message from Sofa: {}".format(message))
# print("Received message from c++ %s" % message)

# Do some 'work'
time.sleep(1)

# Send reply back to client
# socket.send(b"Hola cliente, muy bien y tu ?")
# print('Response sent')



The output or the message which arrive to python zmq server is the symbols
characters of the btnState:

(cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python server.py
ZMQ Server listening ...
*Received message from Sofa: b"\x00'\x84)\xff\x7f\x00\x00\x98&*"

Likely, As I in my python server I am representing like  a string message
arrived and I am sending from my c++ client a int btnState data,
then I try convert int to string in  the python server side:

And I replace this line

print("Received message from Sofa: {}".format(message))

by thiis line

print("Received message from c++ %s" % str(message))

But my output is:

(cnvss_test) ➜  Python git:(ZMQCommunication) ✗ python server.py
ZMQ Server listening ...
*Received message from c++ b'\x00^n&\xff\x7f\x00\x00\xd8\\'*
Traceback (most recent call last):
  File "server.py", line 19, in 
message = socket.recv()
  File "zmq/backend/cython/socket.pyx", line 693, in
zmq.backend.cython.socket.Socket.recv
  File "zmq/backend/cython/socket.pyx", line 727, in
zmq.backend.cython.socket.Socket.recv
  File "zmq/backend/cython/socket.pyx", line 150, in
zmq.backend.cython.socket._recv_copy
  File "zmq/backend/cython/socket.pyx", line 145, in
zmq.backend.cython.socket._recv_copy
  File "zmq/backend/cython/checkrc.pxd", line 25, in
zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Operation cannot be accomplished in current state
(cnvss_test) ➜  Python git:(ZMQCommunication) ✗

I follow getting the characters symbolss.
My question is: How to can I represent the arrived message in my python zmq
server to show their content ?
According to this behavior, can I assume that the btnState data is sent to
python server of anyway?

Any support or orientation will be highly appreciated
___
zeromq-dev mailing

Re: [zeromq-dev] FOSDEM 2018 - ZeroMQ gathering/hackaton?

2018-01-24 Thread Benjamin Henrion

On Wed, Jan 24, 2018 at 9:54 PM, Luca Boccassi  wrote:
> On Sun, 2017-08-27 at 18:20 +0100, Luca Boccassi wrote:
>> Hello,
>>
>> FOSDEM 2018 dates have been announced, it will be on Saturday 3rd and
>> Sunday 4th of February. [1]
>>
>> Would people be interested in replicating last year's 2-days ZMQ
>> gathering/hackaton before FOSDEM?
>>
>> Benjamin, would the hacker space we were at be available again?
>>
>> Kind regards,
>> Luca Boccassi
>>
>> [1] https://fosdem.org/2018/
>
> Hey all,
>
> Just a week to go!
>
> If you have proposed an idea for the hackaton, and it requires some
> non-trivial setup or dependencies, could you please consider preparing
> a short how-to before the event and post it as a comment on the page?
>
> http://zeromq.org/event:zeromq-pre-fosdem-hackaton-thu-1-fri-2-feb-2018
>
> This way we can more easily and quickly bring who is interested up to
> speed.
>
> I'll go first - I've built Ubuntu/Debian packages for DPDK and over the
> weekend I'll prepare a quick howto.

Some practical things:

1. I will be there on Friday the whole day, on Thursday I will have to
work and I will come to open the doors of HSBXL early in the morning
(like 8am), and come back in the afternoon. There are 4 doors/gates to
pass, so I would need someone to be there with me to take the keys. I
will try to go there the day before, and warm up the place;
2. On the friday evening at 6PM, there is a Kubernetes meetup with 40
people, so I will need the help of you guys to move stuff around
(tables, seats, etc...), as I broke my back last year and I cannot
lift heavy things;

See you next week!

-- 
Benjamin Henrion (zoobab)
Email: zoobab at gmail.com
Mobile: +32-484-566109
Web: http://www.zoobab.com
FFII.org Brussels
"In July 2005, after several failed attempts to legalise software
patents in Europe, the patent establishment changed its strategy.
Instead of explicitly seeking to sanction the patentability of
software, they are now seeking to create a central European patent
court, which would establish and enforce patentability rules in their
favor, without any possibility of correction by competing courts or
democratically elected legislators."
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] FOSDEM 2018 - ZeroMQ gathering/hackaton?

2018-01-24 Thread Luca Boccassi

On Sun, 2017-08-27 at 18:20 +0100, Luca Boccassi wrote:
> Hello,
> 
> FOSDEM 2018 dates have been announced, it will be on Saturday 3rd and
> Sunday 4th of February. [1]
> 
> Would people be interested in replicating last year's 2-days ZMQ
> gathering/hackaton before FOSDEM?
> 
> Benjamin, would the hacker space we were at be available again?
> 
> Kind regards,
> Luca Boccassi
> 
> [1] https://fosdem.org/2018/

Hey all,

Just a week to go!

If you have proposed an idea for the hackaton, and it requires some
non-trivial setup or dependencies, could you please consider preparing
a short how-to before the event and post it as a comment on the page?

http://zeromq.org/event:zeromq-pre-fosdem-hackaton-thu-1-fri-2-feb-2018

This way we can more easily and quickly bring who is interested up to
speed.

I'll go first - I've built Ubuntu/Debian packages for DPDK and over the
weekend I'll prepare a quick howto.

-- 
Kind regards,
Luca Boccassi

signature.asc
Description: This is a digitally signed message part
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Pattern for two clients and one server

Re: [zeromq-dev] Communication between C++ zmq client and Python zmq Server

[zeromq-dev] Communication between C++ zmq client and Python zmq Server

Re: [zeromq-dev] FOSDEM 2018 - ZeroMQ gathering/hackaton?

Re: [zeromq-dev] FOSDEM 2018 - ZeroMQ gathering/hackaton?

5 matches

Site Navigation

Mail list logo

Footer information