[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation URL: https://github.com/apache/celix/pull/172#discussion_r402526337 ## File path: bundles/pubsub/pubsub_spi/include/pubsub_utils_url.h ## @@ -0,0 +1,55 @@ +/* Review comment: Sounds good. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation URL: https://github.com/apache/celix/pull/172#discussion_r402521453 ## File path: bundles/pubsub/pubsub_spi/include/pubsub_utils_url.h ## @@ -0,0 +1,55 @@ +/* Review comment: @pnoltes @rbulter what do you guys think? Does it make sense to move this shared code to something like pubsub_shared? I agree it is generic enough to have in here, but I don't thinks it should be part of spi. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation URL: https://github.com/apache/celix/pull/172#discussion_r401802801 ## File path: bundles/pubsub/pubsub_spi/include/pubsub_utils_url.h ## @@ -0,0 +1,55 @@ +/* Review comment: Ah ok, I get it, since it is used by the other Admins as well I guess this makes sense. My first impression was that since it is in the spi, it would be a requirement for other admins as well. Thinking out loud now, does it make sense to move these kind of functions to another directory? Technically it is not a SPI. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401555821
##
File path: bundles/pubsub/pubsub_spi/include/pubsub_protocol.h
##
@@ -28,39 +28,78 @@
typedef struct pubsub_protocol_header pubsub_protocol_header_t;
+/**
+ * The protocol header structure, contains the information about the message
payload and metadata
+ */
struct pubsub_protocol_header {
-unsigned int msgId;
-unsigned short msgMajorVersion;
-unsigned short msgMinorVersion;
-
-unsigned int payloadSize;
-unsigned int metadataSize;
+ /** message payload identification attributes */
+unsigned int msgId; /*!< Message id of the payload */
Review comment:
Even so, I don't see much value in any kind of doc that just re-iterates
what the member already specifies.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401410476
##
File path: bundles/pubsub/pubsub_admin_zmq/src/pubsub_zmq_topic_sender.c
##
@@ -562,33 +566,38 @@ static int psa_zmq_topicPublicationSend(void* handle,
unsigned int msgTypeId, co
bool sendOk;
if (bound->parent->zeroCopyEnabled) {
+int flags = 0;
zmq_msg_t msg1; // Header
zmq_msg_t msg2; // Payload
zmq_msg_t msg3; // Metadata
void *socket = zsock_resolve(sender->zmq.socket);
-zmq_msg_init_data(&msg1, headerData, headerLength,
psa_zmq_freeMsg, bound);
//send header
-int rc = zmq_msg_send(&msg1, socket, ZMQ_SNDMORE);
+zmq_msg_init_data(&msg1, headerData, headerLength,
psa_zmq_freeMsg, bound);
+if ((payloadLength > 0) || (metadataLength > 0)) {
+ flags = ZMQ_SNDMORE;
Review comment:
Does it make sense to only send a header, without any payload?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401424069
##
File path:
bundles/pubsub/pubsub_protocol_wire_v1/src/pubsub_wire_protocol_impl.c
##
@@ -65,19 +65,40 @@ celix_status_t
pubsubProtocol_destroy(pubsub_protocol_wire_v1_t* protocol) {
return status;
}
+celix_status_t pubsubProtocol_getHeaderSize(void* handle, size_t *length) {
+*length = sizeof(int) * 5 + sizeof(short) * 2; // header + sync + version
= 24
+
+return CELIX_SUCCESS;
+}
+
+celix_status_t pubsubProtocol_getHeaderBufferSize(void* handle, size_t
*length) {
+return pubsubProtocol_getHeaderSize(handle, length);
+}
+
+celix_status_t pubsubProtocol_getSyncHeaderSize(void* handle, size_t *length)
{
+*length = sizeof(int);
+return CELIX_SUCCESS;
+}
+
celix_status_t pubsubProtocol_getSyncHeader(void* handle, void *syncHeader) {
-for (int i = 0; i < 5; ++i) {
-((char *) syncHeader)[i] = '\0';
-}
writeInt(syncHeader, 0, PROTOCOL_WIRE_SYNC);
-
return CELIX_SUCCESS;
}
+celix_status_t pubsubProtocol_isMessageSegmentationSupported(void* handle,
bool *isSupported) {
+*isSupported = false;
+return CELIX_SUCCESS;
+}
celix_status_t pubsubProtocol_encodeHeader(void *handle,
pubsub_protocol_message_t *message, void **outBuffer, size_t *outLength) {
celix_status_t status = CELIX_SUCCESS;
+// Get HeaderSize
+size_t headerSize = 0;
+pubsubProtocol_getHeaderSize(handle, &headerSize);
Review comment:
What is the difference between headersize and headerbuffersize, I've read
the doc at the header, but don't quite get it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401412687
##
File path: bundles/pubsub/pubsub_admin_zmq/src/pubsub_zmq_topic_sender.c
##
@@ -562,33 +566,38 @@ static int psa_zmq_topicPublicationSend(void* handle,
unsigned int msgTypeId, co
bool sendOk;
if (bound->parent->zeroCopyEnabled) {
+int flags = 0;
zmq_msg_t msg1; // Header
zmq_msg_t msg2; // Payload
zmq_msg_t msg3; // Metadata
void *socket = zsock_resolve(sender->zmq.socket);
-zmq_msg_init_data(&msg1, headerData, headerLength,
psa_zmq_freeMsg, bound);
//send header
-int rc = zmq_msg_send(&msg1, socket, ZMQ_SNDMORE);
+zmq_msg_init_data(&msg1, headerData, headerLength,
psa_zmq_freeMsg, bound);
+if ((payloadLength > 0) || (metadataLength > 0)) {
+ flags = ZMQ_SNDMORE;
+}
+
+int rc = zmq_msg_send(&msg1, socket, flags);
if (rc == -1) {
L_WARN("Error sending header msg. %s", strerror(errno));
zmq_msg_close(&msg1);
}
-//send header
-if (rc > 0) {
+//send Payload
+if (rc > 0 && payloadLength > 0) {
zmq_msg_init_data(&msg2, payloadData, payloadLength,
psa_zmq_freeMsg, bound);
-int flags = 0;
if (metadataLength > 0) {
-flags = ZMQ_SNDMORE;
+ flags = ZMQ_SNDMORE;
Review comment:
flags should be reset to 0, since at line 578 it is already set to SNDMORE
if this line is reached.
I thinks "if (metadataLength == 0) flags = 0;" should work..
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation URL: https://github.com/apache/celix/pull/172#discussion_r401548808 ## File path: bundles/pubsub/pubsub_spi/include/pubsub_utils.h ## @@ -165,10 +165,11 @@ bool pubsub_utils_matchEndpoint( * * @param bundle The bundle where the properties reside. * @param topic The topic name. + * @param scope The scope name. * @param isPublishertrue if the topic properties for a publisher should be found. * @return The topic properties if found or NULL. */ -celix_properties_t* pubsub_utils_getTopicProperties(const celix_bundle_t *bundle, const char *topic, bool isPublisher); +celix_properties_t* pubsub_utils_getTopicProperties(const celix_bundle_t *bundle, const char *topic, const char *scope, bool isPublisher); Review comment: Just a suggestion, placing scope before topic makes more sense imo. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation URL: https://github.com/apache/celix/pull/172#discussion_r401551814 ## File path: bundles/pubsub/test/test/test_endpoint_runner.cc ## @@ -18,22 +18,54 @@ */ #include "celix_api.h" Review comment: We are moving to GTest, not sure if it makes sense to do this now as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation URL: https://github.com/apache/celix/pull/172#discussion_r401549272 ## File path: bundles/pubsub/pubsub_spi/include/pubsub_utils_url.h ## @@ -0,0 +1,55 @@ +/* Review comment: Why this generic file? Isn't this admin specific? PS doesn't have to dictate how an admin uses urls. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation URL: https://github.com/apache/celix/pull/172#discussion_r401551326 ## File path: bundles/pubsub/pubsub_spi/src/pubsub_utils_match.c ## @@ -227,6 +228,7 @@ double pubsub_utils_matchSubscriber( pubsub_get_topic_properties_data_t data; data.isPublisher = false; data.topic = celix_properties_get(svcProperties, PUBSUB_SUBSCRIBER_TOPIC, NULL); +data.scope = celix_properties_get(svcProperties, PUBSUB_SUBSCRIBER_SCOPE, PUBSUB_SUBSCRIBER_SCOPE_DEFAULT); Review comment: Just a heads up, with PR 181, the default scope is actually NULL, not "default". Only in places where a value is needed, a value is used (discovery and internal map key). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401411569
##
File path: bundles/pubsub/pubsub_admin_zmq/src/pubsub_zmq_topic_sender.c
##
@@ -529,12 +526,17 @@ static int psa_zmq_topicPublicationSend(void* handle,
unsigned int msgTypeId, co
if (status == CELIX_SUCCESS /*ser ok*/) {
pubsub_protocol_message_t message;
-message.payload.payload = serializedOutput;
-message.payload.length = serializedOutputLen;
+message.payload.payload = NULL;
+message.payload.length = 0;
+message.metadata.metadata = NULL;
void *payloadData = NULL;
size_t payloadLength = 0;
-entry->protSer->encodePayload(entry->protSer->handle, &message,
&payloadData, &payloadLength);
+if (serializedOutput) {
+ message.payload.payload = serializedOutput->iov_base;
Review comment:
I see a lot of 2 space indentations, this should be 4.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401550373
##
File path: bundles/pubsub/pubsub_spi/src/pubsub_utils.c
##
@@ -143,12 +143,17 @@ celix_properties_t
*pubsub_utils_getTopicProperties(const celix_bundle_t *bundle
bundle_getEntry((celix_bundle_t *)bundle, ".", &bundleRoot);
if (bundleRoot != NULL) {
-asprintf(&topicPropertiesPath,
"%s/META-INF/topics/%s/%s.properties", bundleRoot, isPublisher? "pub":"sub",
topic);
+asprintf(&topicPropertiesPath,
"%s/META-INF/topics/%s/%s.%s,properties", bundleRoot, isPublisher? "pub":"sub",
topic, scope);
Review comment:
Same here, I would expect the scope to be before the topic, as it also is in
the discovery entry.
Also, typo in the name, "...%s.%s,properties", that comma should be a point.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401550690
##
File path: bundles/pubsub/pubsub_spi/src/pubsub_utils.c
##
@@ -143,12 +143,17 @@ celix_properties_t
*pubsub_utils_getTopicProperties(const celix_bundle_t *bundle
bundle_getEntry((celix_bundle_t *)bundle, ".", &bundleRoot);
if (bundleRoot != NULL) {
-asprintf(&topicPropertiesPath,
"%s/META-INF/topics/%s/%s.properties", bundleRoot, isPublisher? "pub":"sub",
topic);
+asprintf(&topicPropertiesPath,
"%s/META-INF/topics/%s/%s.%s,properties", bundleRoot, isPublisher? "pub":"sub",
topic, scope);
topic_props = celix_properties_load(topicPropertiesPath);
+
if (topic_props == NULL) {
-printf("PubSub: Could not load properties for %s on topic %s.
Searched location %s, bundleId=%ld\n", isPublisher?
"publication":"subscription", topic, topicPropertiesPath, bundleId);
+ free(topicPropertiesPath);
+ asprintf(&topicPropertiesPath,
"%s/META-INF/topics/%s/%s.properties", bundleRoot, isPublisher ? "pub" : "sub",
topic);
+ topic_props = celix_properties_load(topicPropertiesPath);
+ if (topic_props == NULL) {
+printf("PubSub: Could not load properties for %s on topic %s.
Searched location %s, bundleId=%ld\n", isPublisher ? "publication" :
"subscription", topic, topicPropertiesPath, bundleId);
Review comment:
Scope can be added to this message.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401544042
##
File path:
bundles/pubsub/pubsub_serializer_avrobin/src/pubsub_avrobin_serializer_impl.c
##
@@ -49,10 +49,10 @@ struct pubsub_avrobin_serializer {
log_helper_t *loghelper;
};
-static celix_status_t pubsubMsgAvrobinSerializer_serialize(void *handle, const
void *msg, void **out, size_t *outLen);
-static celix_status_t pubsubMsgAvrobinSerializer_deserialize(void *handle,
const void *input, size_t inputLen, void **out);
-static void pubsubMsgAvrobinSerializer_freeMsg(void *handle, void *msg);
-
+static celix_status_t pubsubMsgAvrobinSerializer_serialize(void *handle, const
void *msg, struct iovec** output, size_t* outputIovLen);
Review comment:
Shouldn't all those changes be in a different PR?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
[GitHub] [celix] abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and add interfacing for VectorIoSerialisation
abroekhuis commented on a change in pull request #172: Refactor TcpAdmin and
add interfacing for VectorIoSerialisation
URL: https://github.com/apache/celix/pull/172#discussion_r401545652
##
File path: bundles/pubsub/pubsub_spi/include/pubsub_protocol.h
##
@@ -28,39 +28,78 @@
typedef struct pubsub_protocol_header pubsub_protocol_header_t;
+/**
+ * The protocol header structure, contains the information about the message
payload and metadata
+ */
struct pubsub_protocol_header {
-unsigned int msgId;
-unsigned short msgMajorVersion;
-unsigned short msgMinorVersion;
-
-unsigned int payloadSize;
-unsigned int metadataSize;
+ /** message payload identification attributes */
+unsigned int msgId; /*!< Message id of the payload */
Review comment:
Curious about this notation: !< ... does it have some meaning? If not, does
it make sense to clean this up? msgId with a comment "message id" etc doesn't
add much imo.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
