On 20 May 2015 at 10:25, Maxim Uvarov <[email protected]> wrote:
> Hello Ola,
>
> please find some replays bellow.
>
> On 19 May 2015 at 01:03, Ola Liljedahl <[email protected]> wrote:
>>
>> As promised, here is my first attempt at a standalone API for IPC - inter
>> process communication in a shared nothing architecture (message passing
>> between processes which do not share memory).
>>
>> Currently all definitions are in the file ipc.h but it is possible to
>> break out some message/event related definitions (everything from
>> odp_ipc_sender) in a separate file message.h. This would mimic the
>> packet_io.h/packet.h separation.
>>
>> The semantics of message passing is that sending a message to an endpoint
>> will always look like it succeeds. The appearance of endpoints is
>> explicitly
>> notified through user-defined messages specified in the odp_ipc_resolve()
>> call. Similarly, the disappearance (e.g. death or otherwise lost
>> connection)
>> is also explicitly notified through user-defined messages specified in the
>> odp_ipc_monitor() call. The send call does not fail because the addressed
>> endpoints has disappeared.
>>
>> Messages (from endpoint A to endpoint B) are delivered in order. If
>> message
>> N sent to an endpoint is delivered, then all messages <N have also been
>> delivered. Message delivery does not guarantee actual processing by the
>> recipient. End-to-end acknowledgements (using messages) should be used if
>> this guarantee is important to the user.
>>
>> IPC endpoints can be seen as interfaces (taps) to an internal reliable
>> multidrop network where each endpoint has a unique address which is only
>> valid for the lifetime of the endpoint. I.e. if an endpoint is destroyed
>> and then recreated (with the same name), the new endpoint will have a
>> new address (eventually endpoints addresses will have to be recycled but
>> not for a very long time). Endpoints names do not necessarily have to be
>> unique.
>>
>> Signed-off-by: Ola Liljedahl <[email protected]>
>> ---
>> (This document/code contribution attached is provided under the terms of
>> agreement LES-LTM-21309)
>>
>> include/odp/api/ipc.h | 261
>> ++++++++++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 261 insertions(+)
>> create mode 100644 include/odp/api/ipc.h
>>
>> diff --git a/include/odp/api/ipc.h b/include/odp/api/ipc.h
>> new file mode 100644
>> index 0000000..3395a34
>> --- /dev/null
>> +++ b/include/odp/api/ipc.h
>> @@ -0,0 +1,261 @@
>> +/* Copyright (c) 2015, Linaro Limited
>> + * All rights reserved.
>> + *
>> + * SPDX-License-Identifier: BSD-3-Clause
>> + */
>> +
>> +
>> +/**
>> + * @file
>> + *
>> + * ODP IPC API
>> + */
>> +
>> +#ifndef ODP_API_IPC_H_
>> +#define ODP_API_IPC_H_
>> +
>> +#ifdef __cplusplus
>> +extern "C" {
>> +#endif
>> +
>> +/** @defgroup odp_ipc ODP IPC
>> + * @{
>> + */
>> +
>> +/**
>> + * @typedef odp_ipc_t
>> + * ODP IPC handle
>> + */
>> +
>> +/**
>> + * @typedef odp_ipc_msg_t
>> + * ODP IPC message handle
>> + */
>> +
>> +
>> +/**
>> + * @def ODP_IPC_ADDR_SIZE
>> + * Size of the address of an IPC endpoint
>> + */
>> +
>> +/**
>> + * Create IPC endpoint
>> + *
>> + * @param name Name of local IPC endpoint
>> + * @param pool Pool for incoming messages
>> + *
>> + * @return IPC handle on success
>> + * @retval ODP_IPC_INVALID on failure and errno set
>> + */
>> +odp_ipc_t odp_ipc_create(const char *name, odp_pool_t pool);
>> +
>
>
> Will it be separate pool or you want to attach existing pool to your ipc?
It should be a pool with events of type messages (ODP_EVENT_MESSAGE)
so in practice it will likely be a separate pool.
>
>
>>
>> +/**
>> + * Destroy IPC endpoint
>> + *
>> + * @param ipc IPC handle
>> + *
>> + * @retval 0 on success
>> + * @retval <0 on failure
>> + */
>> +int odp_ipc_destroy(odp_ipc_t ipc);
>> +
>> +/**
>> + * Set the default input queue for an IPC endpoint
>> + *
>> + * @param ipc IPC handle
>> + * @param queue Queue handle
>> + *
>> + * @retval 0 on success
>> + * @retval <0 on failure
>> + */
>> +int odp_ipc_inq_setdef(odp_ipc_t ipc, odp_queue_t queue);
>> +
>> +/**
>> + * Remove the default input queue
>> + *
>> + * Remove (disassociate) the default input queue from an IPC endpoint.
>> + * The queue itself is not touched.
>> + *
>> + * @param ipc IPC handle
>> + *
>> + * @retval 0 on success
>> + * @retval <0 on failure
>> + */
>> +int odp_ipc_inq_remdef(odp_ipc_t ipc);
>> +
>> +/**
>> + * Resolve endpoint by name
>> + *
>> + * Look up an existing or future endpoint by name.
>> + * When the endpoint exists, return the specified message with the
>> endpoint
>> + * as the sender.
>> + *
>> + * @param ipc IPC handle
>> + * @param name Name to resolve
>> + * @param msg Message to return
>> + */
>> +void odp_ipc_resolve(odp_ipc_t ipc,
>> + const char *name,
>> + odp_ipc_msg_t msg);
>> +
>> +/**
>> + * Monitor endpoint
>> + *
>> + * Monitor an existing (potentially already dead) endpoint.
>> + * When the endpoint is dead, return the specified message with the
>> endpoint
>> + * as the sender.
>> + *
>> + * Unrecognized or invalid endpoint addresses are treated as dead
>> endpoints.
>> + *
>> + * @param ipc IPC handle
>> + * @param addr Address of monitored endpoint
>> + * @param msg Message to return
>> + */
>> +void odp_ipc_monitor(odp_ipc_t ipc,
>> + const uint8_t addr[ODP_IPC_ADDR_SIZE],
>> + odp_ipc_msg_t msg);
>> +
>> +/**
>> + * Send message
>> + *
>> + * Send a message to an endpoint (which may already be dead).
>> + * Message delivery is ordered and reliable. All (accepted) messages will
>> be
>> + * delivered up to the point of endpoint death or lost connection.
>> + * Actual reception and processing is not guaranteed (use end-to-end
>> + * acknowledgements for that).
>> + * Monitor the remote endpoint to detect death or lost connection.
>> + *
>> + * @param ipc IPC handle
>> + * @param msg Message to send
>> + * @param addr Address of remote endpoint
>> + *
>> + * @retval 0 on success
>> + * @retval <0 on error
>> + */
>> +int odp_ipc_send(odp_ipc_t ipc,
>> + odp_ipc_msg_t msg,
>> + const uint8_t addr[ODP_IPC_ADDR_SIZE]);
>> +
>> +/**
>> + * Get address of sender (source) of message
>> + *
>> + * @param msg Message handle
>> + * @param addr Address of sender endpoint
>> + */
>> +void odp_ipc_sender(odp_ipc_msg_t msg,
>> + uint8_t addr[ODP_IPC_ADDR_SIZE]);
>> +
>> +/**
>> + * Message data pointer
>> + *
>> + * Return a pointer to the message data
>> + *
>> + * @param msg Message handle
>> + *
>> + * @return Pointer to the message data
>> + */
>> +void *odp_ipc_data(odp_ipc_msg_t msg);
>> +
>> +/**
>> + * Message data length
>> + *
>> + * Return length of the message data.
>> + *
>> + * @param msg Message handle
>> + *
>> + * @return Message length
>> + */
>> +uint32_t odp_ipc_length(const odp_ipc_msg_t msg);
>> +
>> +/**
>> + * Set message length
>> + *
>> + * Set length of the message data.
>> + *
>> + * @param msg Message handle
>> + * @param len New length
>> + *
>> + * @retval 0 on success
>> + * @retval <0 on error
>> + */
>> +int odp_ipc_reset(const odp_ipc_msg_t msg, uint32_t len);
>
>
> reset usually stand to reset to some default value. In that case it's better
> to be odp_ipc_len_set(msg, len).
Well I copied this from packet.h where odp_packet_reset(pkt, len)
seemed to be equivalent to what I want to achieve but I agree that
odp_ipc_len_set() is more direct. I can change.
> And reset is just void inline function to reset to default value
> odp_ipc_reset(msg).
> And len might be is not needed here because you transfer events. So just get
The IPC mechanism transfers messages. And different messages have
different lengths.
> size from event.
The message size specified when creating a message pool that is
associated with an IPC endpoint is basically an MTU (the largest
message size that can be sent or received).
>
> Might be something like:
>
> msg.event = ev; // len will be get from event
> msg.remote = addr;
I don't understand what you are trying to describe here.
>
>
>
>>
>> +
>> +/**
>> + * Allocate message
>> + *
>> + * Allocate a message of a specific size.
>> + *
>> + * @param pool Message pool to allocate message from
>> + * @param len Length of the allocated message
>> + *
>> + * @return IPC message handle on success
>> + * @retval ODP_IPC_MSG_INVALID on failure and errno set
>> + */
>> +odp_ipc_msg_t odp_ipc_alloc(odp_pool_t pool, uint32_t len);
>> +
>
>
> is len sizeof(odp_ipc_msg_t) here?
No. odp_ipc_msg_t is a handle to a message, just like odp_packet_t is
a handle to a packet.
The implementation of odp_ipc_msg_t is platform specific and thus
sizeof(odp_ipc_msg_t) as well (but 32 or 64 bits is a good guess).
The len parameter in odp_ipc_alloc() specifies the current/actual
length of the message which depends on what the application wants to
use that specific message for.
>
>
>>
>> +/**
>> + * Free message
>> + *
>> + * Free message back to the message pool it was allocated from.
>> + *
>> + * @param msg Handle of message to free
>> + */
>> +void odp_ipc_free(odp_ipc_msg_t msg);
>> +
>> +/**
>> + * Get message handle from event
>> + *
>> + * Converts an ODP_EVENT_MESSAGE type event to a message.
>> + *
>> + * @param ev Event handle
>> + *
>> + * @return Message handle
>> + *
>> + * @see odp_event_type()
>> + */
>> +odp_ipc_msg_t odp_message_from_event(odp_event_t ev);
>> +
>> +/**
>> + * Convert message handle to event
>> + *
>> + * @param msg Message handle
>> + *
>> + * @return Event handle
>> + */
>> +odp_event_t odp_message_to_event(odp_ipc_msg_t msg);
>> +
>
>
> Most of functions look like pktio. But that version of IPC looks like fully
> software implementation.
I think some SoC's could use HW mechanism to transfer messages (which
are basically buffers with individual lengths) between endpoints (e.g.
different address spaces).
> Idea is good - send and receive odp events.
Send and receive ODP *messages*, not any type of ODP events.
> It looks like odp_ipc_t ipc
> object also connected to
> some odp_pktio_t where actual packets will be sent. And it will be
> interesting to make queues interface
> work with that ipc.
>
> Might be:
> odp_pktio_t pktio can have ipc atribute like
>
> odp_ipc_pktio_set(odp_ipc_t ipc, odp_pktio_t pktio);
Now you are down to implementation details. Why should this be part of
the public API?
>
> then queue created for that pktio can send/recv packets from outside.
I think you can already do this with the current packet_io API. No
need to conflate this with IPC/message passing.
>
> If I understand right, that functions use queue only for ipc messages:
> int odp_ipc_inq_setdef(odp_ipc_t ipc, odp_queue_t queue);
> int odp_ipc_inq_remdef(odp_ipc_t ipc);
You want to be able to receive and dispatch IPC messages just like any
other type of events so incoming messages should be put on queues. The
queue could be specified when the IPC endpoint is created but I copied
the packet_io API where it was possible.
> and they are not linked with outside pktio. Earlier you said that propose
> to send some packets to some internal tap device.
No not packets, messages. And the reference to "tap" was in a general
sense, not related to Linux tap interfaces.
> And in that case tap
> device can be pktio. And then this pktio-tap connected to ipc object to
> dispatch events from it.
>
> And probably we need better names for our ipc implementation. Mine can be
> odp_pktio_ipc and your can be odp_ipcmsg_ because the propose is different.
Actually I don't understand why your packet pipes need to change the
ODP API. They are just pktio interfaces with different semantics. By
using different pktio device names (which anyway are platform
specific), you create packet pipes instead of opening physical network
interfaces. But they behave the same from the user perspective. Or?
>
>
>
>
>>
>> +/**
>> + * Get printable value for an odp_ipc_t
>> + *
>> + * @param ipc IPC handle to be printed
>> + * @return uint64_t value that can be used to print/display this
>> + * handle
>> + *
>> + * @note This routine is intended to be used for diagnostic purposes
>> + * to enable applications to generate a printable value that represents
>> + * an odp_ipc_t handle.
>> + */
>> +uint64_t odp_ipc_to_u64(odp_ipc_t ipc);
>> +
>> +/**
>> + * Get printable value for an odp_ipc_msg_t
>> + *
>> + * @param msg Message handle to be printed
>> + * @return uint64_t value that can be used to print/display this
>> + * handle
>> + *
>> + * @note This routine is intended to be used for diagnostic purposes
>> + * to enable applications to generate a printable value that represents
>> + * an odp_ipc_msg_t handle.
>> + */
>> +uint64_t odp_ipc_msg_to_u64(odp_ipc_msg_t msg);
>> +
>> +/**
>> + * @}
>> + */
>> +
>> +#ifdef __cplusplus
>> +}
>> +#endif
>> +
>> +#endif
>> --
>> 1.9.1
>
>
>
> After writing replay I came with idea why not build one pktio based on other
> pktio?
> In that case eth0_pktio = ocp_create_pktio("eth0") can be used for external
> packets exchange and
> ipc_eth0_pktio = odp_create_pktio(eth0_pktio, ..).
> So read from eth0_pktio will get only outside packets and read from
> ipc_eth0_pktio can deliver only
> ipc packets.
There are no IPC packets, just IPC messages.
> In that case you do not need to implement transport layer and
> just do functionality above
> on current pktio.
IPC/message passing has different semantics (e.g. reliable/in-order)
from network interfaces.
IPC/message passing is for communication between local entities (e.g.
applications on the same host), not for communicating with external
entities over some physical network.
So I see several serious incompatibilities and I see no reason to
build IPC/message passing on top of some physical network interface
and its representation in ODP.
>
> Thanks,
> Maxim.
>
>
>
>
>
>>
>>
>> _______________________________________________
>> lng-odp mailing list
>> [email protected]
>> https://lists.linaro.org/mailman/listinfo/lng-odp
>
>
_______________________________________________
lng-odp mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/lng-odp
