Re: Sync dwctwo(4) with NetBSD
On Wed, Jul 14, 2021 at 05:57:16PM +0200, Marcus Glocker wrote: > Also this new version includes jsg@ suggestion, and revert the static > bits back to what we have in our current code base. My RPI3b+ ist running this OK with mue(4), ugold(4) and an additonal powered USB hub. Thank You. -m
Re: rsync getopt_long cleanup
ok benno@
much better as the list grows
Claudio Jeker(cje...@diehard.n-r-g.com) on 2021.07.13 20:06:39 +0200:
> I never really liked the getopt_long definitions in rsync. Too much magic
> and chaos.
>
> This moves the table out of main to gain some more space and to make it a
> proper read-only object. Because of this struct opts also needs to become
> a global but that is OK.
>
> Clean up the required_argument options that have no short from. Instead of
> small numbers use some defines and make the values larger than any char
> value (I chose 1000 and up).
>
> Fix --no-motd, it is just a flag setting a value. So just use the
> getopt_long() method for doing that.
>
> Sort the options alphabetically with the exception of no-XYZ options which
> I added below the XYZ option itself.
>
> IMO the result is better than what was there before.
> --
> :wq Claudio
>
> Index: main.c
> ===
> RCS file: /cvs/src/usr.bin/rsync/main.c,v
> retrieving revision 1.55
> diff -u -p -r1.55 main.c
> --- main.c30 Jun 2021 13:10:04 - 1.55
> +++ main.c13 Jul 2021 17:54:13 -
> @@ -269,61 +269,67 @@ fargs_parse(size_t argc, char *argv[], s
> return f;
> }
>
> +static struct optsopts;
> +
> +#define OP_ADDRESS 1000
> +#define OP_PORT 1001
> +#define OP_RSYNCPATH 1002
> +#define OP_TIMEOUT 1003
> +#define OP_VERSION 1004
> +
> +const struct option lopts[] = {
> +{ "address", required_argument, NULL,OP_ADDRESS },
> +{ "archive", no_argument,NULL, 'a' },
> +{ "compress",no_argument,NULL, 'z' },
> +{ "del", no_argument,&opts.del, 1 },
> +{ "delete", no_argument,&opts.del, 1 },
> +{ "devices", no_argument,&opts.devices, 1 },
> +{ "no-devices", no_argument,&opts.devices, 0 },
> +{ "dry-run", no_argument,&opts.dry_run, 1 },
> +{ "group", no_argument,&opts.preserve_gids,1 },
> +{ "no-group",no_argument,&opts.preserve_gids,0 },
> +{ "help",no_argument,NULL, 'h' },
> +{ "links", no_argument,&opts.preserve_links, 1 },
> +{ "no-links",no_argument,&opts.preserve_links, 0 },
> +{ "no-motd", no_argument,&opts.no_motd, 1 },
> +{ "numeric-ids", no_argument,&opts.numeric_ids, 1 },
> +{ "owner", no_argument,&opts.preserve_uids,1 },
> +{ "no-owner",no_argument,&opts.preserve_uids,0 },
> +{ "perms", no_argument,&opts.preserve_perms, 1 },
> +{ "no-perms",no_argument,&opts.preserve_perms, 0 },
> +{ "port",required_argument, NULL,OP_PORT
> },
> +{ "recursive", no_argument,&opts.recursive,1 },
> +{ "no-recursive",no_argument,&opts.recursive,0 },
> +{ "rsh", required_argument, NULL,'e' },
> +{ "rsync-path", required_argument, NULL,OP_RSYNCPATH },
> +{ "sender", no_argument,&opts.sender, 1 },
> +{ "server", no_argument,&opts.server, 1 },
> +{ "specials",no_argument,&opts.specials, 1 },
> +{ "no-specials", no_argument,&opts.specials, 0 },
> +{ "timeout", required_argument, NULL,OP_TIMEOUT },
> +{ "times", no_argument,&opts.preserve_times, 1 },
> +{ "no-times",no_argument,&opts.preserve_times, 0 },
> +{ "verbose", no_argument,&verbose, 1 },
> +{ "no-verbose", no_argument,&verbose, 0 },
> +{ "version", no_argument,NULL, OP_VERSION },
> +{ NULL, 0, NULL, 0 }
> +};
> +
> int
> main(int argc, char *argv[])
> {
> - struct opts opts;
> pid_tchild;
> int fds[2], sd = -1, rc, c, st, i;
> struct sess sess;
> struct fargs*fargs;
> char**args;
> const char *errstr;
> - const struct option lopts[] = {
> - { "port", required_argument, NULL,3 },
> - { "rsh",required_argument, NULL,'e' },
> - { "rsync-path", required_argument, NULL,1 },
> - { "sender", no_argument,&opts.sender, 1 },
> - { "server", no_argument,&opts.server, 1 },
> - { "dry-run",no_argument,&opts.dry_run, 1 },
> - { "version",no_argument,NULL, 2 },
> - { "archive",no_argument,NULL, 'a'
Re: Sync dwctwo(4) with NetBSD
> Date: Wed, 14 Jul 2021 12:20:08 +0200 > From: Marcus Glocker > > On Wed, 14 Jul 2021 11:53:50 +0200 (CEST) > Mark Kettenis wrote: > > > > Date: Mon, 12 Jul 2021 11:56:28 +1000 > > > From: Jonathan Gray > > > > > > On Sun, Jul 11, 2021 at 05:55:14PM +0200, Marcus Glocker wrote: > > > > dwctwo(4) on the Raspberry Pi 3 has some issues today. Basically > > > > uhub2 doesn't work which shows in: > > > > > > > > - mue(4) doesn't attach on uhub2 port 1. > > > > - Removable devices don't attach on uhub2 port 2 and 3. > > > > > > on the 3b+ you mean, smsc(4) is fine on the 3b > > > > > > > > > > > On NetBSD this works fine, but looking for specific fixes in the > > > > NetBSD code base shows quickly that we have grown a big gap > > > > between both code bases. This makes syncing back specific fixes, > > > > if you can identify them at all, very difficult, if not > > > > impossible. > > > > > > > > Therefore I took the path of syncing the entire NetBSD code base > > > > with ours. This fixes the above issues on my Raspberry Pi 3 > > > > which is kind of nice, especially since we can use the integrated > > > > mue(4) Ethernet controller by default now. > > > > > > > > I decided to keep the list_* queue macros from Linux, and use the > > > > Linux headers which are already included in our code base for > > > > drm(4). This just makes future syncing easier. > > > > > > You should not do this. They exist because there are millions of > > > lines of rapidly changing code in drm and only have the barest of > > > what is required for drm. dwc2 is quite small in comparison and > > > I'm sure there are still problems in the drm replacement linux > > > functions. > > > > When I looked at the dwc2 code in the past I learned that replacing > > the Linux list APIs with the interfaces from is not > > trivial. IMHO the Linux list APIs are rather dangerous as they make > > doing list validation almost impossible but they allow some operations > > that the BSD interfaces don't implement. So I think switching back to > > the Linux APIs in this codebase makes sense. But the code should > > probably use its own copy of those interfaces to avoid any unwanted > > interaction with future drm updates as Linux really doesn't have > > stable APIs. > > I'm glad you are highlighting this again, since looking once more at the > porting of the list_* macros just gave me an headache. As you > mentioned correctly, there are certain operations of the list_* macros > which our queue macros don't support, and in the past dwctwo(4) > introduced things like "linked" and "in_freelist" variables to > workaround that, which isn't nice IMO, and turn the code base out of > sync again. > > In my initial version of the synced code I used local copies of the > Linux headers located in sys/dev/usb/dwc2. How about importing the > NetBSD ported Linux headers which are required to sys/dev/usb/dwc2 from > sys/external/bsd/common/include/linux? > > http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/external/bsd/common/include/linux/?only_with_tag=MAIN > > I can shot an adapted diff for that shortly. Something like that might make sense since that is the implementation that the NetBSD code was tested against. I would prefer if you just imported the bits that you actually need, so maybe cut down the list.h file to size and avoid the other header files if you can.
Re: Sync dwctwo(4) with NetBSD
On Wed, 14 Jul 2021 11:53:50 +0200 (CEST) Mark Kettenis wrote: > > Date: Mon, 12 Jul 2021 11:56:28 +1000 > > From: Jonathan Gray > > > > On Sun, Jul 11, 2021 at 05:55:14PM +0200, Marcus Glocker wrote: > > > dwctwo(4) on the Raspberry Pi 3 has some issues today. Basically > > > uhub2 doesn't work which shows in: > > > > > > - mue(4) doesn't attach on uhub2 port 1. > > > - Removable devices don't attach on uhub2 port 2 and 3. > > > > on the 3b+ you mean, smsc(4) is fine on the 3b > > > > > > > > On NetBSD this works fine, but looking for specific fixes in the > > > NetBSD code base shows quickly that we have grown a big gap > > > between both code bases. This makes syncing back specific fixes, > > > if you can identify them at all, very difficult, if not > > > impossible. > > > > > > Therefore I took the path of syncing the entire NetBSD code base > > > with ours. This fixes the above issues on my Raspberry Pi 3 > > > which is kind of nice, especially since we can use the integrated > > > mue(4) Ethernet controller by default now. > > > > > > I decided to keep the list_* queue macros from Linux, and use the > > > Linux headers which are already included in our code base for > > > drm(4). This just makes future syncing easier. > > > > You should not do this. They exist because there are millions of > > lines of rapidly changing code in drm and only have the barest of > > what is required for drm. dwc2 is quite small in comparison and > > I'm sure there are still problems in the drm replacement linux > > functions. > > When I looked at the dwc2 code in the past I learned that replacing > the Linux list APIs with the interfaces from is not > trivial. IMHO the Linux list APIs are rather dangerous as they make > doing list validation almost impossible but they allow some operations > that the BSD interfaces don't implement. So I think switching back to > the Linux APIs in this codebase makes sense. But the code should > probably use its own copy of those interfaces to avoid any unwanted > interaction with future drm updates as Linux really doesn't have > stable APIs. I'm glad you are highlighting this again, since looking once more at the porting of the list_* macros just gave me an headache. As you mentioned correctly, there are certain operations of the list_* macros which our queue macros don't support, and in the past dwctwo(4) introduced things like "linked" and "in_freelist" variables to workaround that, which isn't nice IMO, and turn the code base out of sync again. In my initial version of the synced code I used local copies of the Linux headers located in sys/dev/usb/dwc2. How about importing the NetBSD ported Linux headers which are required to sys/dev/usb/dwc2 from sys/external/bsd/common/include/linux? http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/external/bsd/common/include/linux/?only_with_tag=MAIN I can shot an adapted diff for that shortly.
Re: Sync dwctwo(4) with NetBSD
> Date: Mon, 12 Jul 2021 11:56:28 +1000 > From: Jonathan Gray > > On Sun, Jul 11, 2021 at 05:55:14PM +0200, Marcus Glocker wrote: > > dwctwo(4) on the Raspberry Pi 3 has some issues today. Basically uhub2 > > doesn't work which shows in: > > > > - mue(4) doesn't attach on uhub2 port 1. > > - Removable devices don't attach on uhub2 port 2 and 3. > > on the 3b+ you mean, smsc(4) is fine on the 3b > > > > > On NetBSD this works fine, but looking for specific fixes in the NetBSD > > code base shows quickly that we have grown a big gap between both code > > bases. This makes syncing back specific fixes, if you can identify them > > at all, very difficult, if not impossible. > > > > Therefore I took the path of syncing the entire NetBSD code base with > > ours. This fixes the above issues on my Raspberry Pi 3 which is kind > > of nice, especially since we can use the integrated mue(4) Ethernet > > controller by default now. > > > > I decided to keep the list_* queue macros from Linux, and use the Linux > > headers which are already included in our code base for drm(4). This > > just makes future syncing easier. > > You should not do this. They exist because there are millions of > lines of rapidly changing code in drm and only have the barest of what > is required for drm. dwc2 is quite small in comparison and I'm sure > there are still problems in the drm replacement linux functions. When I looked at the dwc2 code in the past I learned that replacing the Linux list APIs with the interfaces from is not trivial. IMHO the Linux list APIs are rather dangerous as they make doing list validation almost impossible but they allow some operations that the BSD interfaces don't implement. So I think switching back to the Linux APIs in this codebase makes sense. But the code should probably use its own copy of those interfaces to avoid any unwanted interaction with future drm updates as Linux really doesn't have stable APIs.
Re: Sync dwctwo(4) with NetBSD
On Mon, 12 Jul 2021 11:56:28 +1000 Jonathan Gray wrote: > On Sun, Jul 11, 2021 at 05:55:14PM +0200, Marcus Glocker wrote: > > dwctwo(4) on the Raspberry Pi 3 has some issues today. Basically > > uhub2 doesn't work which shows in: > > > > - mue(4) doesn't attach on uhub2 port 1. > > - Removable devices don't attach on uhub2 port 2 and 3. > > on the 3b+ you mean, smsc(4) is fine on the 3b Yes of course. Sorry, I didn't distinguish clearly. > > On NetBSD this works fine, but looking for specific fixes in the > > NetBSD code base shows quickly that we have grown a big gap between > > both code bases. This makes syncing back specific fixes, if you > > can identify them at all, very difficult, if not impossible. > > > > Therefore I took the path of syncing the entire NetBSD code base > > with ours. This fixes the above issues on my Raspberry Pi 3 which > > is kind of nice, especially since we can use the integrated mue(4) > > Ethernet controller by default now. > > > > I decided to keep the list_* queue macros from Linux, and use the > > Linux headers which are already included in our code base for > > drm(4). This just makes future syncing easier. > > You should not do this. They exist because there are millions of > lines of rapidly changing code in drm and only have the barest of what > is required for drm. dwc2 is quite small in comparison and I'm sure > there are still problems in the drm replacement linux functions. Hmm, I see. I'll have a try to port the list_* macros back to TAILQ_* without changing the code logic too much, but that could get tricky ... > changing STATIC to static will also mean functions don't show > up in backtraces which isn't what we want. Sure, that can be easily reverted.
Re: Sync dwctwo(4) with NetBSD
On Wed, 14 Jul 2021 07:30:30 +0200 Martin Reindl wrote: > On Sun, Jul 11, 2021 at 05:55:14PM +0200, Marcus Glocker wrote: > > > > Following the entire diff. And of course, general feedback and > > comments? :-) > > > > While I can't comment on the diff itself, I gave it a spin on my > Raspberry Pi 3 Model B Plus Rev 1.3. mue(4) now works and I am happy > to test further diffs. Quick test with an uvideo(4) webcam shows > isochronous transfer still don't work but maybe this can be fixed > down the road? Thanks for testing. Yes, isoc transfers are a different story, and I didn't expect them to work with this version of dwctwo(4). I would also consider this to be looked at in a second step.
Re: bgpd refactor struct prefix
On Tue, Jun 29, 2021 at 12:00:24PM +0200, Claudio Jeker wrote:
> This diff moves the rib_entry pointer re into the union to safe some
> space. For add-path I need to add a few more u_int32_t and that would
> blow the size of struct prefix from 128 to 132 bytes. malloc would round
> that up to 256bytes and that is bad for the struct that is allocted in
> millions in bgpd.
>
> To make this somewhat save introduce PREFIX_FLAG_ADJOUT to mark prefixes
> that live in the adj-rib-out. Those prefixes can not access the re pointer
> also use a wrapper prefix_re() which returns the re pointer or NULL.
> Also add some assertions to make sure that prefixes don't end up in the
> wrong tree.
>
> This change shrinks the struct back to 120bytes and gives me the space
> needed for add-path.
>
> Please test
Ping
--
:wq Claudio
Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.530
diff -u -p -r1.530 rde.c
--- rde.c 25 Jun 2021 09:25:48 - 1.530
+++ rde.c 29 Jun 2021 08:28:33 -
@@ -2298,6 +2298,7 @@ rde_dump_rib_as(struct prefix *p, struct
struct ibuf *wbuf;
struct attr *a;
struct nexthop *nexthop;
+ struct rib_entry*re;
void*bp;
time_t staletime;
size_t aslen;
@@ -2330,7 +2331,8 @@ rde_dump_rib_as(struct prefix *p, struct
rib.origin = asp->origin;
rib.validation_state = p->validation_state;
rib.flags = 0;
- if (p->re != NULL && p->re->active == p)
+ re = prefix_re(p);
+ if (re != NULL && re->active == p)
rib.flags |= F_PREF_ACTIVE;
if (!prefix_peer(p)->conf.ebgp)
rib.flags |= F_PREF_INTERNAL;
@@ -2412,14 +2414,16 @@ static void
rde_dump_filter(struct prefix *p, struct ctl_show_rib_request *req)
{
struct rde_aspath *asp;
+ struct rib_entry*re;
if (!rde_match_peer(prefix_peer(p), &req->neighbor))
return;
asp = prefix_aspath(p);
+ re = prefix_re(p);
if (asp == NULL)/* skip pending withdraw in Adj-RIB-Out */
return;
- if ((req->flags & F_CTL_ACTIVE) && p->re->active != p)
+ if ((req->flags & F_CTL_ACTIVE) && re != NULL && re->active != p)
return;
if ((req->flags & F_CTL_INVALID) &&
(asp->flags & F_ATTR_PARSE_ERR) == 0)
Index: rde.h
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.h,v
retrieving revision 1.240
diff -u -p -r1.240 rde.h
--- rde.h 17 Jun 2021 16:05:26 - 1.240
+++ rde.h 29 Jun 2021 08:33:18 -
@@ -56,10 +56,10 @@ struct rib {
struct filter_head *in_rules_tmp;
u_int rtableid;
u_int rtableid_tmp;
+ enum reconf_action state, fibstate;
+ u_int16_t id;
u_int16_t flags;
u_int16_t flags_tmp;
- u_int16_t id;
- enum reconf_action state, fibstate;
};
#define RIB_ADJ_IN 0
@@ -317,13 +317,13 @@ struct prefix {
union {
struct {
LIST_ENTRY(prefix) rib, nexthop;
+ struct rib_entry*re;
} list;
struct {
RB_ENTRY(prefix) index, update;
} tree;
}entry;
struct pt_entry *pt;
- struct rib_entry*re;
struct rde_aspath *aspath;
struct rde_community*communities;
struct rde_peer *peer;
@@ -338,6 +338,7 @@ struct prefix {
#definePREFIX_FLAG_DEAD0x04/* locked but removed */
#definePREFIX_FLAG_STALE 0x08/* stale entry (graceful
reload) */
#definePREFIX_FLAG_MASK0x0f/* mask for the prefix types */
+#definePREFIX_FLAG_ADJOUT 0x10/* prefix is in the adj-out rib
*/
#definePREFIX_NEXTHOP_LINKED 0x40/* prefix is linked onto
nexthop list */
#definePREFIX_FLAG_LOCKED 0x80/* locked by rib walker */
};
@@ -637,6 +638,14 @@ static inline u_int8_t
prefix_vstate(struct prefix *p)
{
return (p->validation_state & ROA_MASK);
+}
+
+static inline struct rib_entry *
+prefix_re(struct prefix *p)
+{
+ if (p->flags & PREFIX_FLAG_ADJOUT)
+ return NULL;
+ return (p->entry.list.re);
}
voidnexthop_init(u_int32_t);
Index: rde_rib.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde_rib.c,v
retrieving revision 1.222
diff -u -p -r1.222 rde_rib.c
--- rde_rib.c 17 Jun 2021 08:16:04 - 1
