pnoltes commented on code in PR #690: URL: https://github.com/apache/celix/pull/690#discussion_r1407919336
########## bundles/remote_services/README.md: ########## @@ -33,26 +33,25 @@ The topology manager decides which services should be imported and exported acco ### Remote Service Admin -The Remote Service Admin (RSA) provides the mechanisms to import and export services when instructed to do so by the Topology Manager. +The Remote Service Admin (RSA) provides the mechanisms to import and export services when instructed to do so by the Topology Manager. -#### Endpoints and proxies +#### Remote Service Admin DFI -To delegate a *received* method call to the actual service implementation, the RSA uses an "endpoint" bundle, which has all the knowledge about the marshalling and unmarshalling of data for the service. This endpoint bundle is specific to the used RSA implementation, and as such cannot be reused between various RSA implementations. +Provides remote service admin using HTTP and JSON. The serialization is done using [libdfi](../../libs/dfi/README.md) to convert function call information into [JSON representation](https://amdatu.atlassian.net/wiki/spaces/AMDATUDEV/pages/21954571/Amdatu+Remote#AmdatuRemote-AdminHTTP%2FJson). +`libffi` is configured using descriptor files in the bundles. -Invoking a *remote* method is done by using "proxy" bundles. Similar as to endpoints, proxy bundles encapsulate all knowledge to marshall and unmarshall data for a remote method call and as such can not be shared between RSA implementations. - -Both proxy and endpoint bundles are loaded on demand when a service is imported or exported by the RSA. As such, these bundles **must** not be added to the list of "auto started" bundles, but placed in a separate location. By default, `endpoints` is used as location for locating proxy and/or endpoint bundles. - -Note that since endpoints and proxies need to be created manually, one has full control about the handling of specifics of marshalling and unmarshalling data and dealing with exceptions. +| **Bundle** | `rsa_dfi.zip` | Review Comment: I prefer to use the cmake target names, including the celix alias (e.g. the target downstream users use). So in this case I prefer: `Celix::rsa_dfi`. Ideally we should also start prefix our bundle filenames with a `celix_` prefix. e..g. as is done with http admin: ``` add_celix_bundle(http_admin SOURCES src/http_admin.c src/websocket_admin.c src/activator.c src/service_tree.c VERSION 1.0.0 SYMBOLIC_NAME "apache_celix_http_admin" GROUP "Celix/HTTP_admin" NAME "Apache Celix HTTP Admin" FILENAME celix_http_admin # <----- ensure that the bundle zip becomes celix_http_admin.zip ) ``` ########## bundles/remote_services/discovery_zeroconf/README.md: ########## @@ -0,0 +1,73 @@ +--- +title: Discovery Zeroconf +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Discovery Zeroconf + +The `Discovery_zeroconf` is implemented based on [Bonjour](https://github.com/apple-oss-distributions/mDNSResponder), and its operation depends on the mDNS daemon. + +The mapping between celix and mdns services is as follows: + +| **mDNS service** | **celix service** | +|------------------|-------------------------------------------| +| instance name | service name+hash(endpoint uuid) | +| service type | "_celix-rpc._udp"+${custom subtype} | +| domain name | "local" | +| txt record | service properties | +| host | "celix_rpc_dumb_host.local."(It is dummy) | +| port | 50009(It is dummy) | + + +Because We will perform the mDNS query only using link-local multicast, so we set domain name default value "local". Review Comment: ```suggestion The domain name value is set to "local" , because for remote discovery the mDNS query will only use link-local multicast. ``` ########## libs/dfi/README.md: ########## @@ -19,4 +19,243 @@ See the License for the specific language governing permissions and limitations under the License. --> -# Apache Celix Dynamic Function Interface +## Apache Celix Dynamic Function Interface + +Dynamic Function Interface (DFI) is a dynamic interface type implementation based on [libffi](https://en.wikipedia.org/wiki/Libffi). +It can generate dynamic interface types according to the interface description file, and can convert the corresponding +dynamic interface call into a [JSON representation](https://amdatu.atlassian.net/wiki/spaces/AMDATUDEV/pages/21954571/Amdatu+Remote#AmdatuRemote-AdminHTTP%2FJson). It can also convert the JSON representation into a dynamic interface call. + +### Conan Option + build_celix_dfi=True Default is False + +### CMake Option + CELIX_DFI=ON Default is ON + +### Interface Descriptor + +In `libdfi`, we have defined a set of simple interface description languages. Users can use this language to describe interface types, and at runtime, libdfi converts the interface description into a dynamic interface. + +Before introducing the interface description language, let's look at an example of interface description. + +~~~ +:header +type=interface +name=calculator +version=1.0.0 +:annotations +classname=org.example.Calculator +:types +StatsResult={DDD[D average min max input} +:methods +add(DD)D=add(#am=handle;PDD#am=pre;*D)N +~~~ + +As above, the interface description has four sections: the header section, annotations section, types section, and methods section. The format of each section is as follows: + +~~~ + ':header\n' 'Name=Value\n'... + ':annotations\n' 'Name=Value\n'... + ':types\n' 'TypeId=Value\n'... + ':methods\n' 'MethodId=Value\n'... +~~~ + +Among them, the legal characters that can be used in the “name” and “TypeId” include [a-zA-Z0-9_], the legal characters in “MethodId” include [a-zA-Z0-9_] and ".();[{}/". Besides [a-zA-Z0-9], the legal characters in “value” also include ".<>{}[]?;:~!@#$%^&*()_+-=,./'". It's worth noting that there should not be spaces on either side of '=', and each statement must end with a newline(‘\n’). + +For the interface description, its header section must include three elements: type, name, version. The value of "type" should be "interface", "name" is the interface name (service name), the value of "version" should coincide with the version number in the actually used interface header file, and it should conform to [semantic versioning requirements](https://semver.org/). + +#### The Data Types In The Interface Descriptor + +The data types supported by the interface description include: + +- **Simple Types** + + *Type schema*: + + |**Identifier**|B |D |F |I |J |S |V |Z |b | i | j | s |P |t |N | + |---------|---|------|-----|-------|-------|-------|----|--------------|-----|--------|--------|--------|------|------|---| + |**Types**|char|double|float|int32_t|int64_t|int16_t|void|boolean(uint8)|uchar|uint32_t|uint64_t|uint16_t|void *|char *|int| Review Comment: I think the `t` is not really necessary anymore. IIRC I added the `t` for 2 reasons: 1) Although C uses a `char*` as string, in other languages a string type is often a special built-in type. 2) In the beginning of libdfi there was not support to specific if something was a pointer or not, so `t` was needed to specify a `char*`, now `*B` can be used. ########## bundles/remote_services/discovery_zeroconf/README.md: ########## @@ -0,0 +1,73 @@ +--- +title: Discovery Zeroconf +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Discovery Zeroconf + +The `Discovery_zeroconf` is implemented based on [Bonjour](https://github.com/apple-oss-distributions/mDNSResponder), and its operation depends on the mDNS daemon. + +The mapping between celix and mdns services is as follows: + +| **mDNS service** | **celix service** | +|------------------|-------------------------------------------| +| instance name | service name+hash(endpoint uuid) | +| service type | "_celix-rpc._udp"+${custom subtype} | +| domain name | "local" | +| txt record | service properties | +| host | "celix_rpc_dumb_host.local."(It is dummy) | +| port | 50009(It is dummy) | + + +Because We will perform the mDNS query only using link-local multicast, so we set domain name default value "local". + +To reduce the operation of conversion between host name and address info. we set the address info to txt record, and set the host name and port to a dummy value("celix_rpc_dumb_host.local." and "50009"). + +We set the instance name of the mDNS service as `service_name + hash(endpoint uuid)`. If there is a conflict in the instance name, mDNS_daemon will resolve it. Since the maximum size of the mDNS service instance name is 64 bytes, we take the hash of the endpoint uuid here, which also reduces the probability of instance name conflicts. Review Comment: Why is the service name still used, a UUID should be unique enough. Is this for readability/debugging? ########## bundles/remote_services/discovery_zeroconf/diagrams/service_announce_seq.puml: ########## Review Comment: Nice to see the usage of plant UML for some diagrams :smile: ########## bundles/remote_services/remote_service_admin_shm_v2/diagrams/rsa_shm_remote_service_call_seq.png: ########## Review Comment: Again nice to see some sequence diagrams. Especially for a complex sequence like a remote call over the remote service admin :+1: ########## bundles/remote_services/remote_service_admin_shm_v2/README.md: ########## @@ -0,0 +1,86 @@ +--- +title: Remote Service Admin SHM +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Remote Service Admin SHM + +The Celix Remote Service Admin SHM bundle implement OSGi remote service using shared memory. +In order to reuse the (de)serialization features of remote service admin, +the remote service admin will be realized by ‘rsa_shm’ and 'rsa_json_rpc'(see [Remote Service Admin RPC Using JSON](../rsa_rpc_json/README.md)) bundles. +The ‘rsa_shm’ bundle is used to realize ‘remote_service_admin_service_t’ and IPC. +The ‘rsa_json_rpc’ bundle is used to realize (de)serialization and Creating service endpoint/proxy. + +### Supported Platform +- Linux + +### Properties/Configuration + +| **Properties** | **Type** | **Description**| +|----------------|----------|----------------| +| **rsaShmPoolSize** | long | The RSA SHM pool size in bytes. Its value should be greater than 8192 bytes.(default 256KB) | Review Comment: > Its value should be greater than 8192 bytes why? ########## bundles/remote_services/remote_service_admin_shm_v2/README.md: ########## @@ -0,0 +1,86 @@ +--- +title: Remote Service Admin SHM +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Remote Service Admin SHM + +The Celix Remote Service Admin SHM bundle implement OSGi remote service using shared memory. +In order to reuse the (de)serialization features of remote service admin, +the remote service admin will be realized by ‘rsa_shm’ and 'rsa_json_rpc'(see [Remote Service Admin RPC Using JSON](../rsa_rpc_json/README.md)) bundles. +The ‘rsa_shm’ bundle is used to realize ‘remote_service_admin_service_t’ and IPC. +The ‘rsa_json_rpc’ bundle is used to realize (de)serialization and Creating service endpoint/proxy. + +### Supported Platform +- Linux + +### Properties/Configuration + +| **Properties** | **Type** | **Description**| Review Comment: Maybe add a colum "**default**" ########## bundles/remote_services/remote_service_admin_shm_v2/README.md: ########## @@ -0,0 +1,86 @@ +--- +title: Remote Service Admin SHM +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Remote Service Admin SHM + +The Celix Remote Service Admin SHM bundle implement OSGi remote service using shared memory. +In order to reuse the (de)serialization features of remote service admin, +the remote service admin will be realized by ‘rsa_shm’ and 'rsa_json_rpc'(see [Remote Service Admin RPC Using JSON](../rsa_rpc_json/README.md)) bundles. +The ‘rsa_shm’ bundle is used to realize ‘remote_service_admin_service_t’ and IPC. +The ‘rsa_json_rpc’ bundle is used to realize (de)serialization and Creating service endpoint/proxy. Review Comment: ```suggestion The ‘rsa_json_rpc’ bundle is used to realize (de)serialization and creating service endpoint/proxy. ``` ########## bundles/remote_services/rsa_rpc_json/README.md: ########## @@ -0,0 +1,68 @@ +--- +title: Remote Service Admin RPC Using JSON +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Remote Service Admin RPC Using JSON + +Remote service admin RPC does not appear in the OSGI standard document. It is an extraction of the serialization/deserialization functionality from remote service admin. It is part of the remote service admin feature. + +`rsa_json_rpc` is a serialization implementation based on [JSON representation](https://amdatu.atlassian.net/wiki/spaces/AMDATUDEV/pages/21954571/Amdatu+Remote#AmdatuRemote-AdminHTTP%2FJson). It uses `libdfi` to convert function invocation information into JSON messages. The interface description is configured through the description file in the interface consumer/provider. See the [libdfi documentation](../../../libs/dfi/README.md) for the interface description file. + +### Supported Platform +- Linux + +### Properties/Configuration + +| **Properties** | **Type** | **Description**| +|----------------|----------|----------------| +| **RSA_JSON_RPC_LOG_CALLS**| bool | If set to true, the RSA will Log calls info to the file in RSA_JSON_RPC_LOG_CALLS_FILE. Default is false. | +| **RSA_JSON_RPC_LOG_CALLS_FILE**| string | Log file. If RSA_JSON_RPC_LOG_CALLS is enabled, the service calls info will be writen to the file(If restart this bundle, it will truncate file). Default is stdout. | + +### Conan Option + build_rsa_json_rpc=True Default is False + +### CMake Option + RSA_JSON_RPC=ON Default is OFF + +### Software Design + +The implementation of `rsa_json_rpc` includes two parts: 1. Creation of remote service endpoints; 2. Creation of remote service proxies + +- Remote service endpoint: It receives remote JSON_RPC requests and calls the corresponding service instances. +- Remote service proxy: It provides proxy services and serializes service call information (method name, arguments,...) into JSON_RPC requests. + +#### The Process Of Creating And Using A Remote Endpoint + +When a service is exported, RSA can use rsa_json_rpc to create a service endpoint. When a service is called, the service endpoint calls the corresponding service instance after the RPC request is deserialized. The detailed process is as follows diagram: + + + + +#### The Process Of Creating And Using A Remote Proxy + +When a service is imported, RSA can use rsa_json_rpc to create a proxy service factory. When a service is called, the proxy service factory creates the proxy service and gives it to the service caller. Then the service proxy serializes the service call information into an RPC request. And the RPC request will be sent to remote endpoint by RSA. The detailed process is as follows diagram: Review Comment: ```suggestion When a service is imported, RSA can use rsa_json_rpc to create a proxy service factory. When a service is requested, the proxy service factory creates the proxy service and gives it to the service requester. Then when the requested service is called (i.e. the service proxy is called), the service proxy serializes the service call information into an RPC request. And the RPC request will be sent to remote endpoint by RSA. The detailed process is as follows diagram: ``` ########## bundles/remote_services/remote_service_admin_shm_v2/README.md: ########## @@ -0,0 +1,86 @@ +--- +title: Remote Service Admin SHM +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Remote Service Admin SHM + +The Celix Remote Service Admin SHM bundle implement OSGi remote service using shared memory. +In order to reuse the (de)serialization features of remote service admin, +the remote service admin will be realized by ‘rsa_shm’ and 'rsa_json_rpc'(see [Remote Service Admin RPC Using JSON](../rsa_rpc_json/README.md)) bundles. +The ‘rsa_shm’ bundle is used to realize ‘remote_service_admin_service_t’ and IPC. +The ‘rsa_json_rpc’ bundle is used to realize (de)serialization and Creating service endpoint/proxy. + +### Supported Platform +- Linux + +### Properties/Configuration + +| **Properties** | **Type** | **Description**| +|----------------|----------|----------------| +| **rsaShmPoolSize** | long | The RSA SHM pool size in bytes. Its value should be greater than 8192 bytes.(default 256KB) | +| **rsaShmMsgTimeout** | long | The timeout of remote service invocation in seconds. (default 30s) | +| **rsaShmCctIvNum** | long | The maximum concurrent invocations of the same service. If there are more concurrent invocations than its value, service invocation will fail.(default 32) | Review Comment: I did not notice this before, but generally configuration properties follow an all caps naming scheme. This is done because retrieval of config property fall back to environment variables, and environment variable names are de-facto all caps. Looking at the `rsa_shm_constants.h` header I also think the define names are more clear then their currently used property/env variable names: - `RSA_SHM_MEMORY_POOL_SIZE_KEY` - `RSA_SHM_MSG_TIMEOUT_KEY` - `RSA_SHM_MAX_CONCURRENT_INVOCATIONS_KEY` maybe out of scope for this PR, but I think it would be good to update the config property names ########## bundles/remote_services/rsa_rpc_json/README.md: ########## @@ -0,0 +1,68 @@ +--- +title: Remote Service Admin RPC Using JSON +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Remote Service Admin RPC Using JSON + +Remote service admin RPC does not appear in the OSGI standard document. It is an extraction of the serialization/deserialization functionality from remote service admin. It is part of the remote service admin feature. + +`rsa_json_rpc` is a serialization implementation based on [JSON representation](https://amdatu.atlassian.net/wiki/spaces/AMDATUDEV/pages/21954571/Amdatu+Remote#AmdatuRemote-AdminHTTP%2FJson). It uses `libdfi` to convert function invocation information into JSON messages. The interface description is configured through the description file in the interface consumer/provider. See the [libdfi documentation](../../../libs/dfi/README.md) for the interface description file. + +### Supported Platform +- Linux + +### Properties/Configuration + +| **Properties** | **Type** | **Description**| +|----------------|----------|----------------| +| **RSA_JSON_RPC_LOG_CALLS**| bool | If set to true, the RSA will Log calls info to the file in RSA_JSON_RPC_LOG_CALLS_FILE. Default is false. | +| **RSA_JSON_RPC_LOG_CALLS_FILE**| string | Log file. If RSA_JSON_RPC_LOG_CALLS is enabled, the service calls info will be writen to the file(If restart this bundle, it will truncate file). Default is stdout. | + +### Conan Option + build_rsa_json_rpc=True Default is False + +### CMake Option + RSA_JSON_RPC=ON Default is OFF + +### Software Design + +The implementation of `rsa_json_rpc` includes two parts: 1. Creation of remote service endpoints; 2. Creation of remote service proxies + +- Remote service endpoint: It receives remote JSON_RPC requests and calls the corresponding service instances. +- Remote service proxy: It provides proxy services and serializes service call information (method name, arguments,...) into JSON_RPC requests. + +#### The Process Of Creating And Using A Remote Endpoint + +When a service is exported, RSA can use rsa_json_rpc to create a service endpoint. When a service is called, the service endpoint calls the corresponding service instance after the RPC request is deserialized. The detailed process is as follows diagram: + + + + +#### The Process Of Creating And Using A Remote Proxy + +When a service is imported, RSA can use rsa_json_rpc to create a proxy service factory. When a service is called, the proxy service factory creates the proxy service and gives it to the service caller. Then the service proxy serializes the service call information into an RPC request. And the RPC request will be sent to remote endpoint by RSA. The detailed process is as follows diagram: + + + +In the above process, each consumer of the remote service will have a different service proxy, because the service proxy needs to use the interface description file in the consumer (which may be a bundle) to serialize the service call information. Review Comment: > In the same process, there should be one unique copy of service descriptor for each remote service instance. I think this is the key issue. Only 1 unique descriptor for a service per process. Ideally the consumer and provider are also aligned, but this is not always necessary. It is possible for the provider to provide newer version of the service, as long as it is backwards compatible. For example a consumer using the following service: ```C #define FOO_SERVICE_NAME "foo" #define FOO_SERVICE_VERSION "1.0.0" struct { void* handle; int foo(const char* string); } my_service ``` Can use a remote! provided service with the following service: ```C #define FOO_SERVICE_NAME "foo" #define FOO_SERVICE_VERSION "1.1.0" struct { void* handle; int bar(const char* string); int foo(const char* string); } my_service ``` Note that because `bar` is added before `foo` the updated service is not binary backwards compatible inside the same process. But for remote calls, as long as we can identity which method signature is called, different service versions is possible. For the pubsub serializer handler different version is actual handled: https://github.com/apache/celix/blob/rel/celix-2.4.0/bundles/pubsub/pubsub_utils/src/pubsub_serializer_handler.c#L237-L242 I also though there was a check on the literal descriptor content, but apparently not. ########## bundles/remote_services/README.md: ########## @@ -33,26 +33,25 @@ The topology manager decides which services should be imported and exported acco ### Remote Service Admin -The Remote Service Admin (RSA) provides the mechanisms to import and export services when instructed to do so by the Topology Manager. +The Remote Service Admin (RSA) provides the mechanisms to import and export services when instructed to do so by the Topology Manager. -#### Endpoints and proxies +#### Remote Service Admin DFI -To delegate a *received* method call to the actual service implementation, the RSA uses an "endpoint" bundle, which has all the knowledge about the marshalling and unmarshalling of data for the service. This endpoint bundle is specific to the used RSA implementation, and as such cannot be reused between various RSA implementations. +Provides remote service admin using HTTP and JSON. The serialization is done using [libdfi](../../libs/dfi/README.md) to convert function call information into [JSON representation](https://amdatu.atlassian.net/wiki/spaces/AMDATUDEV/pages/21954571/Amdatu+Remote#AmdatuRemote-AdminHTTP%2FJson). +`libffi` is configured using descriptor files in the bundles. -Invoking a *remote* method is done by using "proxy" bundles. Similar as to endpoints, proxy bundles encapsulate all knowledge to marshall and unmarshall data for a remote method call and as such can not be shared between RSA implementations. - -Both proxy and endpoint bundles are loaded on demand when a service is imported or exported by the RSA. As such, these bundles **must** not be added to the list of "auto started" bundles, but placed in a separate location. By default, `endpoints` is used as location for locating proxy and/or endpoint bundles. - -Note that since endpoints and proxies need to be created manually, one has full control about the handling of specifics of marshalling and unmarshalling data and dealing with exceptions. +| **Bundle** | `rsa_dfi.zip` | Review Comment: this comment applies for the rest of the **Bundle** column values. ########## libs/dfi/README.md: ########## @@ -19,4 +19,243 @@ See the License for the specific language governing permissions and limitations under the License. --> -# Apache Celix Dynamic Function Interface +## Apache Celix Dynamic Function Interface + +Dynamic Function Interface (DFI) is a dynamic interface type implementation based on [libffi](https://en.wikipedia.org/wiki/Libffi). +It can generate dynamic interface types according to the interface description file, and can convert the corresponding +dynamic interface call into a [JSON representation](https://amdatu.atlassian.net/wiki/spaces/AMDATUDEV/pages/21954571/Amdatu+Remote#AmdatuRemote-AdminHTTP%2FJson). It can also convert the JSON representation into a dynamic interface call. + +### Conan Option + build_celix_dfi=True Default is False + +### CMake Option + CELIX_DFI=ON Default is ON + +### Interface Descriptor + +In `libdfi`, we have defined a set of simple interface description languages. Users can use this language to describe interface types, and at runtime, libdfi converts the interface description into a dynamic interface. + +Before introducing the interface description language, let's look at an example of interface description. + +~~~ +:header +type=interface +name=calculator +version=1.0.0 +:annotations +classname=org.example.Calculator +:types +StatsResult={DDD[D average min max input} +:methods +add(DD)D=add(#am=handle;PDD#am=pre;*D)N +~~~ + +As above, the interface description has four sections: the header section, annotations section, types section, and methods section. The format of each section is as follows: + +~~~ + ':header\n' 'Name=Value\n'... + ':annotations\n' 'Name=Value\n'... + ':types\n' 'TypeId=Value\n'... + ':methods\n' 'MethodId=Value\n'... +~~~ + +Among them, the legal characters that can be used in the “name” and “TypeId” include [a-zA-Z0-9_], the legal characters in “MethodId” include [a-zA-Z0-9_] and ".();[{}/". Besides [a-zA-Z0-9], the legal characters in “value” also include ".<>{}[]?;:~!@#$%^&*()_+-=,./'". It's worth noting that there should not be spaces on either side of '=', and each statement must end with a newline(‘\n’). + +For the interface description, its header section must include three elements: type, name, version. The value of "type" should be "interface", "name" is the interface name (service name), the value of "version" should coincide with the version number in the actually used interface header file, and it should conform to [semantic versioning requirements](https://semver.org/). + +#### The Data Types In The Interface Descriptor + +The data types supported by the interface description include: + +- **Simple Types** + + *Type schema*: + + |**Identifier**|B |D |F |I |J |S |V |Z |b | i | j | s |P |t |N | + |---------|---|------|-----|-------|-------|-------|----|--------------|-----|--------|--------|--------|------|------|---| + |**Types**|char|double|float|int32_t|int64_t|int16_t|void|boolean(uint8)|uchar|uint32_t|uint64_t|uint16_t|void *|char *|int| + + +- **Complex Types(Struct)** + + *Type schema*: + ~~~ + {[Type]+ [(Name)(SPACE)]+} + ~~~ + *Example*: + ~~~ + {DDII a b c d} + ~~~ + *To C language*: + ~~~ + struct { double a; double b; int c; int d; }; + ~~~ + +- **Sequence Type** + + *Type schema*: + ~~~ + [(type) + ~~~ + *Example*: + ~~~ + [D + ~~~ + *To C language*: + ~~~ + struct { + uint32_t cap; + uint32_t len; + duoble *buf; + }; + ~~~ + +- **Typed Pointer** + + *Type schema*: + ~~~ + *(Type) + ~~~ + *Example*: + ~~~ + *D + ~~~ + *To C language*: + ~~~ + duoble *d; + ~~~ + +- **Reference By Value** + + *Type schema*: + ~~~ + l(name); + ~~~ + *Example*: + ~~~ + MySubType={jDD time d1 d2} + MyType={DDlMySubType; d11 d12 subTypeVal} + ~~~ + *To C language*: + ~~~ + struct MySubType{ + uint64_t time; + double d1; + double d2; + }; + struct MyType { + double d11; + double d12; + struct MySubType subTypeVal; + }; + ~~~ + +- **Pointer Reference** + + *Type schema*: + ~~~ + L(name);//shortcut for *l(name); Review Comment: Given the support for `*`, I think we can drop `L` and `t` so that we do not have alternatives so "say" the same thing. ########## bundles/remote_services/remote_service_admin_shm_v2/README.md: ########## @@ -0,0 +1,86 @@ +--- +title: Remote Service Admin SHM +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +## Remote Service Admin SHM + +The Celix Remote Service Admin SHM bundle implement OSGi remote service using shared memory. +In order to reuse the (de)serialization features of remote service admin, +the remote service admin will be realized by ‘rsa_shm’ and 'rsa_json_rpc'(see [Remote Service Admin RPC Using JSON](../rsa_rpc_json/README.md)) bundles. +The ‘rsa_shm’ bundle is used to realize ‘remote_service_admin_service_t’ and IPC. +The ‘rsa_json_rpc’ bundle is used to realize (de)serialization and Creating service endpoint/proxy. Review Comment: ```suggestion The ‘rsa_json_rpc’ bundle is used to realize (de)serialization and creating service endpoint/proxy. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@celix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
