George Doukas wrote: > Hello everybody... > > I've been working with RTnet over RTAI/LXRT for sometime and now I'm in a > process to integrate ORTE to my work... > > Although I've studied the provided ORTE documentation and some portions of > the source code there are some issues that are not 100% clear to me and > would appreciate if you could help clear them out. > > First of all I assume that ORTE (liborte) supports real-time operation over > RTnet in userspace through LXRT. Is this true? > If this is actually true then in order to use ORTE from real-time LXRT > threads is it required to insmod orte_rt and ortemanager_rt modules kernel > or not? >
Nope. ORTE only includes support to run it the kernel environments of
RTAI or RTLinux/GPL. The makefiles need to be adapted and some
initialisation calls need to be re-organised in order to run over a
RT-userspace libraries.
> I' asking this because I've allready used orte from within LXRT real-time
> threads (after calling rt_make_hard_real_time) without having the 2 modules
> loaded (ortemanager runs as normal process) and this scheme seems to work...
Did you wrap the POSIX calls of liborte onto LXRT? Did you replaced the
socket API calls? A bit more hacking is required in case LXRT should be
used.
Instead, I once (Aug. '05) gave this a try for the POSIX skin of Xenomai
(i.e. RTAI/fusion at that time). I had to change only some parts of the
manager and the examples (init/cleanup stuff...). And I had to fix the
realtime attribute of ORTE thread (well, I had to introduce them). Don't
know if this is fixed now, but Petr (CC'ed) proposed some extension of
ORTE at that time.
This all was possible with few changes to ORTE because the POSIX skin
gets attached to a conforming program just by relinking. The trick:
make CFLAGS="`<xeno-inst-dir>/bin/xeno-config --posix-cflags`" \
LDFLAGS="`<xeno-inst-dir>/bin/xeno-config --posix-ldflags`"
when compiling ORTE for normal Linux. Even RTnet is then automatically
selected as it overloads the UDP sockets in that case. What was and
unfortunately is still missing in Xenomai is rw-lock support or, better,
wrapping. Actually, rw-locks should better be mapped on mutexes in the
RT-case (to optimise the worst case contention, not the average). And
this is what I did by some further hack I don't have at hand. It
extended Xenomai to provide rwlocks as mutexes. But I think today this
should better be done in ORTE directly because the wrapping applies to
other POSIX conforming RTOSes as well.
>
> The reason why I followed this scheme is because I could not load
> ortemanager_rt module... each time I try to insmod ortemanager_rt the system
> freezes!
No need for the kernel module in the userspace scenario, all ORTE work
will be done in userspace.
> I've found an older post about a similar problem and the solution proposed
> was to pass the parameter rtskb_cache_size=64 when loading rtnet.o. That did
> not work for me!
This tweaking of RTnet is required for kernel and userspace scenarios as
ORTE initialises a lot of stuff in RT context (and for this case you
need preallocted buffers in the rtskb_cache). That is partly due to the
RTPS design (which includes some dynamics), but I think it should be
avoidable to some degree. At least one could explicitly leave RT-context
during that allocation (as long as the related routine doesn't hold a
lock at the same time...). I'm not so deep into this anymore, but I see
in my attached hack that the "printf" was introduced to implicitly
switch the mode. Well, it was a hack...
Hope I could provide some useful information, and maybe Petr can add
further comments on plans to improve hard-RT userspace support of ORTE.
Anything happened in this regard in the meantime?
Jan
? build.lnx
? fusion.patch
? patch-rtnet
Index: orte/examples/hello/h_publisher.c
===================================================================
RCS file:
/cvsroot/ocera/ocera/components/comm/eth/orte/orte/examples/hello/h_publisher.c,v
retrieving revision 1.1
diff -u -p -r1.1 h_publisher.c
--- orte/examples/hello/h_publisher.c 23 Feb 2005 10:14:23 -0000 1.1
+++ orte/examples/hello/h_publisher.c 9 Aug 2005 15:05:36 -0000
@@ -73,23 +73,67 @@ publisherCreate(void *arg) {
#ifndef CONFIG_ORTE_RT
+#include <unistd.h>
+#include <stdlib.h>
+#include <signal.h>
+#include <sys/mman.h>
+#include <pthread.h>
+
+pthread_mutex_t mutex; //for wake up
+pthread_cond_t cond; //for wake up
+
+void cleanup_upon_sig(int sig)
+{
+ pthread_cond_signal(&cond);
+}
+
+void *rt_thread(void *arg)
+{
+ ORTEInit();
+ ORTEVerbositySetOptions("ALL.10");
+ d=ORTEDomainAppCreate(ORTE_DEFAULT_DOMAIN,NULL,NULL,ORTE_FALSE);
+ if (!d) {
+ printf("ORTEDomainAppCreate failed!\n");
+ return NULL;
+ }
+ publisherCreate((void*)d);
+
+ pthread_mutex_init(&mutex, NULL);
+ pthread_cond_init(&cond, NULL);
+
+ signal(SIGINT, cleanup_upon_sig);
+ signal(SIGTERM, cleanup_upon_sig);
+ signal(SIGHUP, cleanup_upon_sig);
+
+ pthread_mutex_lock(&mutex);
+ pthread_cond_wait(&cond,&mutex);
+ pthread_mutex_unlock(&mutex);
+
+ pthread_mutex_destroy(&mutex);
+ pthread_cond_destroy(&cond);
+
+ ORTEDomainAppDestroy(d);
+
+ return NULL;
+}
+
int
main(int argc, char *args[]) {
- ORTEInit();
- ORTEVerbositySetOptions("ALL.10");
- d=ORTEDomainAppCreate(ORTE_DEFAULT_DOMAIN,NULL,NULL,ORTE_FALSE);
- if (!d) {
- printf("ORTEDomainAppCreate failed!\n");
- return 0;
- }
- publisherCreate((void*)d);
- while(1)
- ORTESleepMs(1000);
+ struct sched_param param = { .sched_priority = 50 };
+ pthread_attr_t thattr;
+ pthread_t thid;
+
+ mlockall(MCL_CURRENT|MCL_FUTURE);
+
+ pthread_attr_init(&thattr);
+ pthread_attr_setinheritsched(&thattr,PTHREAD_EXPLICIT_SCHED);
+ pthread_attr_setschedpolicy(&thattr,SCHED_FIFO);
+ pthread_attr_setschedparam(&thattr,¶m);
+ pthread_create(&thid,&thattr,&rt_thread,NULL);
- ORTESleepMs(10000);
- printf("finnished!\n");
- ORTEDomainAppDestroy(d);
- return 0;
+ pthread_join(thid, NULL);
+
+ return 0;
}
#else
Index: orte/examples/hello/h_subscriber.c
===================================================================
RCS file:
/cvsroot/ocera/ocera/components/comm/eth/orte/orte/examples/hello/h_subscriber.c,v
retrieving revision 1.1
diff -u -p -r1.1 h_subscriber.c
--- orte/examples/hello/h_subscriber.c 23 Feb 2005 10:14:23 -0000 1.1
+++ orte/examples/hello/h_subscriber.c 9 Aug 2005 15:05:36 -0000
@@ -76,19 +76,67 @@ subscriberCreate(void *arg) {
#ifndef CONFIG_ORTE_RT
+#include <unistd.h>
+#include <stdlib.h>
+#include <signal.h>
+#include <sys/mman.h>
+#include <pthread.h>
+
+pthread_mutex_t mutex; //for wake up
+pthread_cond_t cond; //for wake up
+
+void cleanup_upon_sig(int sig)
+{
+ pthread_cond_signal(&cond);
+}
+
+void *rt_thread(void *arg)
+{
+ ORTEInit();
+ ORTEVerbositySetOptions("ALL.10");
+ d=ORTEDomainAppCreate(ORTE_DEFAULT_DOMAIN,NULL,NULL,ORTE_FALSE);
+ if (!d) {
+ printf("ORTEDomainAppCreate failed!\n");
+ return 0;
+ }
+ subscriberCreate(NULL);
+
+ pthread_mutex_init(&mutex, NULL);
+ pthread_cond_init(&cond, NULL);
+
+ signal(SIGINT, cleanup_upon_sig);
+ signal(SIGTERM, cleanup_upon_sig);
+ signal(SIGHUP, cleanup_upon_sig);
+
+ pthread_mutex_lock(&mutex);
+ pthread_cond_wait(&cond,&mutex);
+ pthread_mutex_unlock(&mutex);
+
+ pthread_mutex_destroy(&mutex);
+ pthread_cond_destroy(&cond);
+
+ ORTEDomainAppDestroy(d);
+
+ return NULL;
+}
+
int
main(int argc, char *args[]) {
- ORTEInit();
- ORTEVerbositySetOptions("ALL.10");
- d=ORTEDomainAppCreate(ORTE_DEFAULT_DOMAIN,NULL,NULL,ORTE_FALSE);
- if (!d) {
- printf("ORTEDomainAppCreate failed!\n");
+ struct sched_param param = { .sched_priority = 50 };
+ pthread_attr_t thattr;
+ pthread_t thid;
+
+ mlockall(MCL_CURRENT|MCL_FUTURE);
+
+ pthread_attr_init(&thattr);
+ pthread_attr_setinheritsched(&thattr,PTHREAD_EXPLICIT_SCHED);
+ pthread_attr_setschedpolicy(&thattr,SCHED_FIFO);
+ pthread_attr_setschedparam(&thattr,¶m);
+ pthread_create(&thid,&thattr,&rt_thread,NULL);
+
+ pthread_join(thid, NULL);
+
return 0;
- }
- subscriberCreate(NULL);
- while (1)
- ORTESleepMs(1000);
- return 0;
}
#else
Index: orte/liborte/ORTEAppRecvThread.c
===================================================================
RCS file:
/cvsroot/ocera/ocera/components/comm/eth/orte/orte/liborte/ORTEAppRecvThread.c,v
retrieving revision 1.1
diff -u -p -r1.1 ORTEAppRecvThread.c
--- orte/liborte/ORTEAppRecvThread.c 23 Feb 2005 10:14:34 -0000 1.1
+++ orte/liborte/ORTEAppRecvThread.c 9 Aug 2005 15:05:37 -0000
@@ -63,6 +63,10 @@ void ORTEAppRecvThread(TaskProp *tp) {
cdrCodec->buffer, //buffer
cdrCodec->buf_len, //max length of message
&des,sizeof(des)); //info from sending host
+ if (RTPS_Codec_len == -1) {
+ debug(22,7) ("ORTEAppRecvThread %s: recvfrom error!
%d\n",TK2S(tp),errno);
+ break;
+ }
debug(22,7) ("ORTEAppRecvThread %s: fired, msg_len:
0x%x\n",TK2S(tp),RTPS_Codec_len);
Index: orte/liborte/ORTEDomain.c
===================================================================
RCS file:
/cvsroot/ocera/ocera/components/comm/eth/orte/orte/liborte/ORTEDomain.c,v
retrieving revision 1.7
diff -u -p -r1.7 ORTEDomain.c
--- orte/liborte/ORTEDomain.c 25 May 2005 19:40:09 -0000 1.7
+++ orte/liborte/ORTEDomain.c 9 Aug 2005 15:05:40 -0000
@@ -25,9 +25,16 @@
void
ORTEDomainRecvThreadStart(TaskProp *tp)
{
+ struct sched_param param = { .sched_priority = 60 };
+ pthread_attr_t thattr;
+
if (tp->terminate) {
tp->terminate=ORTE_FALSE;
- pthread_create(&(tp->thread), NULL,
+ pthread_attr_init(&thattr);
+ pthread_attr_setinheritsched(&thattr,PTHREAD_EXPLICIT_SCHED);
+ pthread_attr_setschedpolicy(&thattr,SCHED_FIFO);
+ pthread_attr_setschedparam(&thattr,¶m);
+ pthread_create(&(tp->thread), &thattr,
(void*)&ORTEAppRecvThread, (void *)tp);
}
}
@@ -36,9 +43,16 @@ ORTEDomainRecvThreadStart(TaskProp *tp)
void
ORTEDomainSendThreadStart(TaskProp *tp)
{
+ struct sched_param param = { .sched_priority = 60 };
+ pthread_attr_t thattr;
+
if (tp->terminate) {
tp->terminate=ORTE_FALSE;
- pthread_create(&(tp->thread), NULL,
+ pthread_attr_init(&thattr);
+ pthread_attr_setinheritsched(&thattr,PTHREAD_EXPLICIT_SCHED);
+ pthread_attr_setschedpolicy(&thattr,SCHED_FIFO);
+ pthread_attr_setschedparam(&thattr,¶m);
+ pthread_create(&(tp->thread), &thattr,
(void*)&ORTEAppSendThread, (void *)tp);
}
}
Index: orte/liborte/debug.c
===================================================================
RCS file: /cvsroot/ocera/ocera/components/comm/eth/orte/orte/liborte/debug.c,v
retrieving revision 1.6
diff -u -p -r1.6 debug.c
--- orte/liborte/debug.c 12 Apr 2005 19:58:36 -0000 1.6
+++ orte/liborte/debug.c 9 Aug 2005 15:05:40 -0000
@@ -43,7 +43,8 @@ FILE *debug_log=NULL; /* NULL */
/*********************************************************************/
/* functions */
-#ifdef CONFIG_ORTE_RT
+//#ifdef CONFIG_ORTE_RT
+#if 1
static const char *
debug_log_time(void) {
struct timespec time;
Index: orte/liborte/htimerNtp.c
===================================================================
RCS file:
/cvsroot/ocera/ocera/components/comm/eth/orte/orte/liborte/htimerNtp.c,v
retrieving revision 1.6
diff -u -p -r1.6 htimerNtp.c
--- orte/liborte/htimerNtp.c 23 Feb 2005 10:14:35 -0000 1.6
+++ orte/liborte/htimerNtp.c 9 Aug 2005 15:05:41 -0000
@@ -232,7 +232,8 @@ htimerUnicastSendUserData_run_expired(OR
NtpTime
getActualNtpTime(void) {
NtpTime result;
-#ifndef CONFIG_ORTE_RT
+//#ifndef CONFIG_ORTE_RT
+#if 0
struct timeval time;
gettimeofday(&time,NULL);
Index: orte/liborte/sock.c
===================================================================
RCS file: /cvsroot/ocera/ocera/components/comm/eth/orte/orte/liborte/sock.c,v
retrieving revision 1.7
diff -u -p -r1.7 sock.c
--- orte/liborte/sock.c 23 Feb 2005 10:14:35 -0000 1.7
+++ orte/liborte/sock.c 9 Aug 2005 15:05:41 -0000
@@ -45,6 +45,7 @@ sock_finish(void) {
/*********************************************************************/
int
sock_init_udp(sock_t *sock) {
+printf("creating sock in nrt\n");
sock->fd = socket(AF_INET, SOCK_DGRAM, 0);
if (sock->fd < 0) return -1;
return 0;
@@ -54,6 +55,7 @@ sock_init_udp(sock_t *sock) {
inline void
sock_cleanup(sock_t *sock) {
#if defined(SOCK_BSD)
+printf("closing sock in nrt\n");
close(sock->fd);
#elif defined(SOCK_RTLWIP)
close_socket_np(sock->fd);
Index: orte/manager/ortemanager.c
===================================================================
RCS file:
/cvsroot/ocera/ocera/components/comm/eth/orte/orte/manager/ortemanager.c,v
retrieving revision 1.6
diff -u -p -r1.6 ortemanager.c
--- orte/manager/ortemanager.c 2 Mar 2005 14:26:12 -0000 1.6
+++ orte/manager/ortemanager.c 9 Aug 2005 15:05:42 -0000
@@ -18,7 +18,8 @@
* GNU General Public License for more details.
*
*/
-
+
+#include <sys/mman.h>
#include "orte_all.h"
#ifndef CONFIG_ORTE_RT
@@ -70,24 +71,20 @@ int managerStop(void) {
//Unix daemon support
pthread_mutex_t mutex; //for wake up
pthread_cond_t cond; //for wake up
-int cvalue;
+
void sig_usr(int signo) {
if ((signo==SIGTERM) || (signo==SIGINT)) {
- pthread_mutex_lock(&mutex);
- cvalue=1;
pthread_cond_signal(&cond);
- pthread_mutex_unlock(&mutex);
}
}
void waitForEndingCommand(void) {
pthread_mutex_init(&mutex, NULL);
pthread_cond_init(&cond, NULL);
- cvalue=0;
signal(SIGTERM,sig_usr);
signal(SIGINT,sig_usr);
+ signal(SIGHUP,sig_usr);
pthread_mutex_lock(&mutex);
- while(cvalue==0)
- pthread_cond_wait(&cond,&mutex);
+ pthread_cond_wait(&cond,&mutex);
pthread_mutex_unlock(&mutex);
pthread_mutex_destroy(&mutex);
pthread_cond_destroy(&cond);
@@ -137,7 +134,35 @@ static void usage(void) {
printf(" -h, --help this usage screen\n");
}
+void *init(void *arg)
+{
+ d=ORTEDomainMgrCreate(domain,&dp,events,ORTE_TRUE);
+ if (!d)
+ exit(1);
+
+ #ifdef CONFIG_ORTE_UNIX
+ if (orteDaemon)
+ daemonInit();
+ #endif
+
+ ORTEDomainStart(d,ORTE_TRUE,ORTE_FALSE,ORTE_FALSE,ORTE_FALSE,ORTE_TRUE);
+ #ifndef CONFIG_ORTE_UNIX
+ while(1) ORTESleepMs(1000);
+ #endif
+
+ #ifdef CONFIG_ORTE_UNIX
+ waitForEndingCommand();
+ ORTEDomainMgrDestroy(d);
+ if (events)
+ free(events);
+ #endif
+ return NULL;
+}
+
int main(int argc,char *argv[]) {
+ struct sched_param param = { .sched_priority = 1 };
+ pthread_attr_t thattr;
+ pthread_t thid;
#if defined HAVE_GETOPT_LONG || defined HAVE_GETOPT_LONG_ORTE
static struct option long_opts[] = {
{ "peer",1,0, 'p' },
@@ -242,7 +267,7 @@ int main(int argc,char *argv[]) {
exit(0);
}
- d=ORTEDomainMgrCreate(domain,&dp,events,ORTE_TRUE);
+/* d=ORTEDomainMgrCreate(domain,&dp,events,ORTE_TRUE);
if (!d)
exit(1);
@@ -251,7 +276,7 @@ int main(int argc,char *argv[]) {
daemonInit();
#endif
- ORTEDomainStart(d,ORTE_TRUE,ORTE_FALSE,ORTE_FALSE,ORTE_FALSE,ORTE_TRUE);
+
ORTEDomainStart(d,ORTE_TRUE,ORTE_FALSE,ORTE_FALSE,ORTE_FALSE,ORTE,ORTE_FALSE,ORTE_FALSE,ORTE_FALSE,ORTE_TRUE);
#ifndef CONFIG_ORTE_UNIX
while(1) ORTESleepMs(1000);
#endif
@@ -261,7 +286,15 @@ int main(int argc,char *argv[]) {
ORTEDomainMgrDestroy(d);
if (events)
free(events);
- #endif
+ #endif*/
+ mlockall(MCL_CURRENT|MCL_FUTURE);
+
+ pthread_attr_init(&thattr);
+ pthread_attr_setinheritsched(&thattr,PTHREAD_EXPLICIT_SCHED);
+ pthread_attr_setschedpolicy(&thattr,SCHED_FIFO);
+ pthread_attr_setschedparam(&thattr,¶m);
+ pthread_create(&thid,&thattr,&init,NULL);
+ pthread_join(thid, NULL);
exit(0);
}
signature.asc
Description: OpenPGP digital signature
