Re: RTEMS5 and file descriptors
Thank you Joel for all of the hints.
I find the maximum number - 64 - of file descriptors somehow low when
compared with Linux and RTEMS 4
(RTEMS 4 did not have the restriction with FD_SETSIZE).
Anyhow, my application is EPICS base and most of the open file
descriptors refer to the clients which connect using sockets.
I will place the EPICS channel access gateway "in front" of my RTEMS epics
server to narrow down the number of clients to just one.
This is just a work around but should satisfy the problem.
Best Regards
Mirek
pon., 17 paź 2022 o 16:20 Chris Johns napisał(a):
> On 18/10/2022 9:19 am, Miroslaw Dach wrote:
> >>AFAIK you'd have to patch the header in the C Library when building the
> tools
> > using the RSB to have a possible clean solution. Editing the installed
> header
> > would be uncool.
> > I see , I thought that it is somehow simpler thing.
> >>How many descriptors do you need? And will you be using select()?
> > I need maximum 128 file descriptors to be used.
>
> The issue is a little more complicated because of the way the fd number are
> allocated. RTEMS implements not reusing fd number once free (closed)
> preferring
> to move to an unused fd number until all numbers have been used. The idea
> is to
> try and catch the accidental continued use of an fd after close when an
> open
> follows.
>
> If you allocate 200 descriptors the following select code will work:
>
> /* RTEMS entry point */
> void Init(void) {
> /* start networking */
> int fd = socket(...);
> FD_ZERO();
> FD_SET(fd, );
> nfds = sd + 1;
> r = select(nfds, , NULL, NULL, NULL);
> /* blah blah */
> }
>
> while this will not:
>
> /* RTEMS entry point */
> void Init(void) {
> /* use up all the fd spots select has by default */
> for (i = 0; i < 65; i++) {
> fd = open("afile", ...);
> close(fd);
> }
> /* start networking */
> fd = socket(...);
> FD_ZERO();
> /* out of range access */
> FD_SET(fd, );
> nfds = sd + 1;
> rv = select(nfds, , NULL, NULL, NULL);
> /* blah blah */
> }
>
> If you control the code in quesiton and can make changes the following are
> some
> suggestions:
>
> 1. Consider a thread per descriptor. Select is slow
>
> 2. Look at kqueue, it is a better interface for this type of blocking
>
> 3. Look at the following select work around ..
> https://git.rtems.org/rtems/tree/cpukit/mghttpd/mongoose.c#n1522
>
> Chris
>
___
users mailing list
users@rtems.org
http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On 20/10/2022 4:03 am, Michael Davidsaver wrote: > On 10/17/22 22:50, Chris Johns wrote: >> On 18/10/2022 4:42 pm, Sebastian Huber wrote: >>> On 18/10/2022 06:15, Chris Johns wrote: On 18/10/2022 2:22 pm, Michael Davidsaver wrote: > On 10/17/22 16:20, Chris Johns wrote: >> 2. Look at kqueue, it is a better interface for this type of blocking > Maybe not relevant in Miroslaw's application, but I've found > that the RTEMS kqueue implementation doesn't notify when a > TCP connection is closed by reset. I think this is a lack > of NOTE_EOF *. Thanks. I cannot find a ticket for this? Do you know if one has been created? > > Not by me. I ran into this incidentally while testing some > new networking code using libevent, which helped me towards > finding: > > https://www.mail-archive.com/freebsd-hackers@freebsd.org/msg43808.html > > Which leads with a reasonably clear statement that kqueue > is/was not 100% functionally equivalent to select()/poll(). > At that point I configured libevent to use poll() and > continued with my own troubleshooting. Then unfortunately, > I forgot about it until I read this thread. > > https://github.com/mdavidsaver/pvxs/blob/6ee82fac6533d6551b18aa489cb263adc1333018/src/evhelper.cpp#L171-L178 > > That freebsd thread is 19 years old, and I haven't spent the > time to investigate what happened since then. > > libevent has: > >> #ifdef NOTE_EOF >> /* Make it behave like select() and poll() */ >> if (filter == EVFILT_READ) >> out->fflags = NOTE_EOF; >> #endif > https://github.com/libevent/libevent/blob/8f47d8de281b877450474734594fdc0a60ee35d1/kqueue.c#L193-L197 > > >>> This looks like a general FreeBSD limitation. > > This was my general assessment at the time. > > >> Is NOTE_EOF the same as EV_EOF? I noticed EV_EOF in the FreeBSD man page for >> kqueue. > > I'm no expert on the kqueue mechanism, but I think not. > > https://man.openbsd.org/kqueue.2 > > This openbsd manpage mentions both, although 'NOTE_EOF' is > only mentioned in passing, with no definition given. > > Some searching leads me to think that EV_EOF is a "flag" > while NOTE_EOF is an "fflag" (filter flag). I have not > investigated the precise distinction between the two. Thanks for the response. It is unusual compatibility between implementations of kqueue is not a major concern. Chris ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On 10/17/22 22:50, Chris Johns wrote: On 18/10/2022 4:42 pm, Sebastian Huber wrote: On 18/10/2022 06:15, Chris Johns wrote: On 18/10/2022 2:22 pm, Michael Davidsaver wrote: On 10/17/22 16:20, Chris Johns wrote: 2. Look at kqueue, it is a better interface for this type of blocking Maybe not relevant in Miroslaw's application, but I've found that the RTEMS kqueue implementation doesn't notify when a TCP connection is closed by reset. I think this is a lack of NOTE_EOF *. Thanks. I cannot find a ticket for this? Do you know if one has been created? Not by me. I ran into this incidentally while testing some new networking code using libevent, which helped me towards finding: https://www.mail-archive.com/freebsd-hackers@freebsd.org/msg43808.html Which leads with a reasonably clear statement that kqueue is/was not 100% functionally equivalent to select()/poll(). At that point I configured libevent to use poll() and continued with my own troubleshooting. Then unfortunately, I forgot about it until I read this thread. https://github.com/mdavidsaver/pvxs/blob/6ee82fac6533d6551b18aa489cb263adc1333018/src/evhelper.cpp#L171-L178 That freebsd thread is 19 years old, and I haven't spent the time to investigate what happened since then. libevent has: #ifdef NOTE_EOF /* Make it behave like select() and poll() */ if (filter == EVFILT_READ) out->fflags = NOTE_EOF; #endif https://github.com/libevent/libevent/blob/8f47d8de281b877450474734594fdc0a60ee35d1/kqueue.c#L193-L197 This looks like a general FreeBSD limitation. This was my general assessment at the time. Is NOTE_EOF the same as EV_EOF? I noticed EV_EOF in the FreeBSD man page for kqueue. I'm no expert on the kqueue mechanism, but I think not. https://man.openbsd.org/kqueue.2 This openbsd manpage mentions both, although 'NOTE_EOF' is only mentioned in passing, with no definition given. Some searching leads me to think that EV_EOF is a "flag" while NOTE_EOF is an "fflag" (filter flag). I have not investigated the precise distinction between the two. ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On 18/10/2022 4:42 pm, Sebastian Huber wrote: > On 18/10/2022 06:15, Chris Johns wrote: >> On 18/10/2022 2:22 pm, Michael Davidsaver wrote: >>> On 10/17/22 16:20, Chris Johns wrote: 2. Look at kqueue, it is a better interface for this type of blocking >>> Maybe not relevant in Miroslaw's application, but I've found >>> that the RTEMS kqueue implementation doesn't notify when a >>> TCP connection is closed by reset. I think this is a lack >>> of NOTE_EOF *. >> Thanks. I cannot find a ticket for this? Do you know if one has been created? > > This looks like a general FreeBSD limitation. > Is NOTE_EOF the same as EV_EOF? I noticed EV_EOF in the FreeBSD man page for kqueue. Chris ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On 18/10/2022 06:15, Chris Johns wrote: On 18/10/2022 2:22 pm, Michael Davidsaver wrote: On 10/17/22 16:20, Chris Johns wrote: 2. Look at kqueue, it is a better interface for this type of blocking Maybe not relevant in Miroslaw's application, but I've found that the RTEMS kqueue implementation doesn't notify when a TCP connection is closed by reset. I think this is a lack of NOTE_EOF *. Thanks. I cannot find a ticket for this? Do you know if one has been created? This looks like a general FreeBSD limitation. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On 18/10/2022 2:22 pm, Michael Davidsaver wrote: > On 10/17/22 16:20, Chris Johns wrote: >> 2. Look at kqueue, it is a better interface for this type of blocking > > Maybe not relevant in Miroslaw's application, but I've found > that the RTEMS kqueue implementation doesn't notify when a > TCP connection is closed by reset. I think this is a lack > of NOTE_EOF *. Thanks. I cannot find a ticket for this? Do you know if one has been created? Chris ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On 10/17/22 16:20, Chris Johns wrote: 2. Look at kqueue, it is a better interface for this type of blocking Maybe not relevant in Miroslaw's application, but I've found that the RTEMS kqueue implementation doesn't notify when a TCP connection is closed by reset. I think this is a lack of NOTE_EOF *. The poll() implementation seems to handle reset as expected. * https://github.com/libevent/libevent/blob/8f47d8de281b877450474734594fdc0a60ee35d1/kqueue.c#L193-L197 ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On 18/10/2022 9:19 am, Miroslaw Dach wrote:
>>AFAIK you'd have to patch the header in the C Library when building the tools
> using the RSB to have a possible clean solution. Editing the installed header
> would be uncool.
> I see , I thought that it is somehow simpler thing.
>>How many descriptors do you need? And will you be using select()?
> I need maximum 128 file descriptors to be used.
The issue is a little more complicated because of the way the fd number are
allocated. RTEMS implements not reusing fd number once free (closed) preferring
to move to an unused fd number until all numbers have been used. The idea is to
try and catch the accidental continued use of an fd after close when an open
follows.
If you allocate 200 descriptors the following select code will work:
/* RTEMS entry point */
void Init(void) {
/* start networking */
int fd = socket(...);
FD_ZERO();
FD_SET(fd, );
nfds = sd + 1;
r = select(nfds, , NULL, NULL, NULL);
/* blah blah */
}
while this will not:
/* RTEMS entry point */
void Init(void) {
/* use up all the fd spots select has by default */
for (i = 0; i < 65; i++) {
fd = open("afile", ...);
close(fd);
}
/* start networking */
fd = socket(...);
FD_ZERO();
/* out of range access */
FD_SET(fd, );
nfds = sd + 1;
rv = select(nfds, , NULL, NULL, NULL);
/* blah blah */
}
If you control the code in quesiton and can make changes the following are some
suggestions:
1. Consider a thread per descriptor. Select is slow
2. Look at kqueue, it is a better interface for this type of blocking
3. Look at the following select work around ..
https://git.rtems.org/rtems/tree/cpukit/mghttpd/mongoose.c#n1522
Chris
___
users mailing list
users@rtems.org
http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
Hi Joel, Thank you for your quick answer. >AFAIK you'd have to patch the header in the C Library when building the tools using the RSB to have a possible clean solution. Editing the installed header would be uncool. I see , I thought that it is somehow simpler thing. >How many descriptors do you need? And will you be using select()? I need maximum 128 file descriptors to be used. Best Regards Mirek pon., 17 paź 2022 o 11:39 Joel Sherrill napisał(a): > > > On Mon, Oct 17, 2022, 12:23 PM Miroslaw Dach > wrote: > >> Hello, >> >> I have followed the instruction to change the limit of File Descriptors >> higher than 64: >> >> https://docs.rtems.org/branches/master/user/migration/v4_11-to-v5.html >> chapter 13.1.3. File Descriptors >> >> Is it required to rebuild the RTEMS5 kernel with the >> macro FD_SETSIZE set to the higher value then 64 or is it enough to >> set it along with CONFIGURE_MAXIMUM_FILE_DESCRIPTORS when the >> application is built? >> > > If you go above 64 for configured file descriptors, FD_SETSIZE used by > select() does not automatically change. That is set in the C Library > headers which are considered constant when RTEMS is built. > > AFAIK you'd have to patch the header in the C Library when building the > tools using the RSB to have a possible clean solution. Editing the > installed header would be uncool. > > How many descriptors do you need? And will you be using select()? > > --joel > > > >> Best Regards >> Mirek >> ___ >> users mailing list >> users@rtems.org >> http://lists.rtems.org/mailman/listinfo/users > > ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
Re: RTEMS5 and file descriptors
On Mon, Oct 17, 2022, 12:23 PM Miroslaw Dach wrote: > Hello, > > I have followed the instruction to change the limit of File Descriptors > higher than 64: > > https://docs.rtems.org/branches/master/user/migration/v4_11-to-v5.html > chapter 13.1.3. File Descriptors > > Is it required to rebuild the RTEMS5 kernel with the > macro FD_SETSIZE set to the higher value then 64 or is it enough to > set it along with CONFIGURE_MAXIMUM_FILE_DESCRIPTORS when the application > is built? > If you go above 64 for configured file descriptors, FD_SETSIZE used by select() does not automatically change. That is set in the C Library headers which are considered constant when RTEMS is built. AFAIK you'd have to patch the header in the C Library when building the tools using the RSB to have a possible clean solution. Editing the installed header would be uncool. How many descriptors do you need? And will you be using select()? --joel > Best Regards > Mirek > ___ > users mailing list > users@rtems.org > http://lists.rtems.org/mailman/listinfo/users ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
RTEMS5 and file descriptors
Hello, I have followed the instruction to change the limit of File Descriptors higher than 64: https://docs.rtems.org/branches/master/user/migration/v4_11-to-v5.html chapter 13.1.3. File Descriptors Is it required to rebuild the RTEMS5 kernel with the macro FD_SETSIZE set to the higher value then 64 or is it enough to set it along with CONFIGURE_MAXIMUM_FILE_DESCRIPTORS when the application is built? Best Regards Mirek ___ users mailing list users@rtems.org http://lists.rtems.org/mailman/listinfo/users
