OT: getting the protocol family of a file descriptor
Hey all! I'm currently in the need to get the protocol family that was used to create a socket (and passed via a unix domain socket to another program), and I've not really come up with a proper scheme other than to use getsockname and retrieve sa_family from the resulting socket (which currently matches the socket domain and historically has, but why take the chances ;-)). Is there any other better way to get at the domain (protocol family) of a socket? -- Heiko Wundram Product Application Development - Office Germany - EXPO PARK HANNOVER Beenic Networks GmbH Mailänder Straße 2 30539 Hannover Fon+49 511 / 590 935 - 15 Fax+49 511 / 590 935 - 29 Mobil +49 172 / 43 737 34 Mail [EMAIL PROTECTED] Beenic Networks GmbH - Sitz der Gesellschaft: Hannover Geschäftsführer: Jorge Delgado Registernummer: HRB 61869 Registergericht: Amtsgericht Hannover ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
Aryeh M. Friedman wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Adrian Penisoara wrote: Hi, Side-topic, if you bear with me: if you were to choose again what to use as source revision control system (VCS) from today's offerings, what would you choose to maintain FreeBSD's sources or a side-off project tracking FreeBSD as base that would allow better teams cooperation and easy code merging between projects/branches ? I'm having to use mercurial. I'm not really enjoying it. works ok for small projects. BSD is a bit big for it. doe work foe offline editing, but loses all your BSD history. probably SVK is the way to go from what I hear. For the moment I am thinking that the top contenders would be Bazaar and Mercurial but I would like to know other (developer) opinions. Aegis aegis.sf.net and devel/aegis... to get it to compile you will need to apply a patch I will send you if you want (and/or use the yet to be committed devel/aegis-devel which does the patch at the cost of failing portlint [installs correctly and all that but has some minor issues that prevent committing as of yet]) currently I am working with the aegis developers so none of the hacks (plus a few other things) are not needed (i.e. no special cases needed for freebsd)... to others reading this is going to be the primary cms/vms/vcs for ports 2.0 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com Free software != Free beer Blog: http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHoXALQi2hk2LEXBARAnLUAKClqkEkOGaE6A5ZkNW/dYeIidpzAACaAkRS ZrJDj6I380VjISP65lVN8ek= =TGs6 -END PGP SIGNATURE- ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: OT: getting the protocol family of a file descriptor
Heiko Wundram (Beenic) wrote: Hey all! I'm currently in the need to get the protocol family that was used to create a socket (and passed via a unix domain socket to another program), and I've not really come up with a proper scheme other than to use getsockname and retrieve sa_family from the resulting socket (which currently matches the socket domain and historically has, but why take the chances ;-)). and pretty much always will. Is there any other better way to get at the domain (protocol family) of a socket? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
Am Donnerstag, 31. Januar 2008 10:03:22 schrieb Julian Elischer: I'm having to use mercurial. I'm not really enjoying it. works ok for small projects. BSD is a bit big for it. doe work foe offline editing, but loses all your BSD history. We're using mercurial pretty much for all of our (100,000+ SLOC) repositories, and I cannot agree that it's only appropriate for small projects. As mercurial is a distributed RCS, the hard part in using it is you have to impose some policies, esp. related to merging changes back into a central repository, which aren't required for centralized systems like CVS and subversion, but from my view, the added benefit for a developer in using a distributed revision control system is well worth the extra effort in writing (and thinking) up the policies once. mercurial (at least by default) doesn't allow you to create remote branches anyway (in pushing back changes to the central store), so the policies you might have are effectly enforced by the system anyway. YMMV, of course, and mercurial has its defects (primary checkout/cloning of a large repository from a central store takes ages, at least over a slow link, the last time I had to do this [but I don't know if any progress has been made there]), but for me, it's been working fine for the daily needs I have as a developer. -- Heiko Wundram Product Application Development ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
I've only tried CVS, Mericurial, Clearcase and a bit of Subversion. And if you don't need IDE integration Mercurial seems to be working pretty good. I just read an article about the new merging and branching support coming in Subversion 1.5 and it looks like it might have some future. The IDE support is probably the best of the modern open source VCS. http://www.javaworld.com/javaworld/jw-01-2008/jw-01-svnmerging.html /Johan On Jan 31, 2008 10:25 AM, Heiko Wundram (Beenic) [EMAIL PROTECTED] wrote: Am Donnerstag, 31. Januar 2008 10:03:22 schrieb Julian Elischer: I'm having to use mercurial. I'm not really enjoying it. works ok for small projects. BSD is a bit big for it. doe work foe offline editing, but loses all your BSD history. We're using mercurial pretty much for all of our (100,000+ SLOC) repositories, and I cannot agree that it's only appropriate for small projects. As mercurial is a distributed RCS, the hard part in using it is you have to impose some policies, esp. related to merging changes back into a central repository, which aren't required for centralized systems like CVS and subversion, but from my view, the added benefit for a developer in using a distributed revision control system is well worth the extra effort in writing (and thinking) up the policies once. mercurial (at least by default) doesn't allow you to create remote branches anyway (in pushing back changes to the central store), so the policies you might have are effectly enforced by the system anyway. YMMV, of course, and mercurial has its defects (primary checkout/cloning of a large repository from a central store takes ages, at least over a slow link, the last time I had to do this [but I don't know if any progress has been made there]), but for me, it's been working fine for the daily needs I have as a developer. -- Heiko Wundram Product Application Development ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
I'm having to use mercurial. I'm not really enjoying it. works ok for small projects. BSD is a bit big for it. doe work foe offline editing, but loses all your BSD history. probably SVK is the way to go from what I hear. Im using mercurial on full FreeBSD trees, curiosity makes me ask where do you the deficiency? Ive had no issues patching, branching, merging, transplanting, tracking vendor updates. The only issue i really had was a import of the full cvs tree ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: more on devstat (missing include)
On Wednesday 30 January 2008 06:09:21 pm Pietro Cerutti wrote:
Hi hackers,
I think that i) sys/devicestat.h should include sys/resource.h or ii)
the man page of devstat(9) should mention that the user himself is
supposed to include sys/resource.h.
Without:
cat devstat.c
#include devstat.h
int main(void)
{
devstat_getnumdevs(NULL);
return (0);
}
gcc -ldevstat -o devstat devstat.c
In file included from devstat.c:1:
/usr/include/devstat.h:131: error: 'CPUSTATES' undeclared here (not in a
function)
Exit 1
I can provide a patch if someone could tell me which one of the proposed
solutions above is the preferred one.
I think this might be best:
Index: devstat.h
===
RCS file: /usr/cvs/src/lib/libdevstat/devstat.h,v
retrieving revision 1.11
diff -u -r1.11 devstat.h
--- devstat.h 22 Jul 2005 17:19:00 - 1.11
+++ devstat.h 31 Jan 2008 10:52:37 -
@@ -32,6 +32,7 @@
#define _DEVSTAT_H
#include sys/cdefs.h
#include sys/devicestat.h
+#include sys/resource.h
#include kvm.h
--
John Baldwin
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: more on devstat (missing include)
John Baldwin wrote:
On Wednesday 30 January 2008 06:09:21 pm Pietro Cerutti wrote:
Hi hackers,
I think that i) sys/devicestat.h should include sys/resource.h or ii)
the man page of devstat(9) should mention that the user himself is
supposed to include sys/resource.h.
Without:
cat devstat.c
#include devstat.h
int main(void)
{
devstat_getnumdevs(NULL);
return (0);
}
gcc -ldevstat -o devstat devstat.c
In file included from devstat.c:1:
/usr/include/devstat.h:131: error: 'CPUSTATES' undeclared here (not in a
function)
Exit 1
I can provide a patch if someone could tell me which one of the proposed
solutions above is the preferred one.
I think this might be best:
Index: devstat.h
===
RCS file: /usr/cvs/src/lib/libdevstat/devstat.h,v
retrieving revision 1.11
diff -u -r1.11 devstat.h
--- devstat.h 22 Jul 2005 17:19:00 - 1.11
+++ devstat.h 31 Jan 2008 10:52:37 -
@@ -32,6 +32,7 @@
#define _DEVSTAT_H
#include sys/cdefs.h
#include sys/devicestat.h
+#include sys/resource.h
#include kvm.h
Good. Are you committing that or I better send in a PR?
--
Pietro Cerutti
PGP Public Key:
http://gahr.ch/pgp
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: OT: getting the protocol family of a file descriptor
Heiko Wundram (Beenic) [EMAIL PROTECTED] writes: I'm currently in the need to get the protocol family that was used to create a socket (and passed via a unix domain socket to another program), and I've not really come up with a proper scheme other than to use getsockname and retrieve sa_family from the resulting socket (which currently matches the socket domain and historically has, but why take the chances ;-)). This is the correct way to do it, I don't understand why you think it might not work in the future. Is there any other better way to get at the domain (protocol family) of a socket? Why should there be a better way? DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
Adrian Penisoara [EMAIL PROTECTED] writes: Side-topic, if you bear with me: if you were to choose again what to use as source revision control system (VCS) from today's offerings, what would you choose to maintain FreeBSD's sources or a side-off project tracking FreeBSD as base that would allow better teams cooperation and easy code merging between projects/branches ? Subversion, hands down. DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: OT: getting the protocol family of a file descriptor
Am Donnerstag, 31. Januar 2008 16:31:32 schrieben Sie: Heiko Wundram (Beenic) [EMAIL PROTECTED] writes: I'm currently in the need to get the protocol family that was used to create a socket (and passed via a unix domain socket to another program), and I've not really come up with a proper scheme other than to use getsockname and retrieve sa_family from the resulting socket (which currently matches the socket domain and historically has, but why take the chances ;-)). This is the correct way to do it, I don't understand why you think it might not work in the future. Is there any other better way to get at the domain (protocol family) of a socket? Why should there be a better way? Just like there is getsockopt(fd,SOL_SOCKET,SO_TYPE), I'd (somehow) assume that there's a getsockopt(fd,SOL_SOCKET,SO_DOMAIN/SO_FAMILY). I can understand that the actual protocol used to create a socket is irrelevant in most of the applications which get sockets from outside (as long as the type they want matches the type of the passed socket), and I've not found any (general [!], of course there's specific ways depending on the PF) way to get at that yet from a file-descriptor, but I can't understand why this also applies to the domain, as for example in case you're trying to format the address a socket is bound/connected to in a user-readable manner, you will have different code depending on the actual address type. Currently, you're basically required to do a getsockname to a struct sockaddr_storage and typecast that to the actual socket addres type based on the ss_family member (to be able to pass it to one of the *_ntop-functions, for example), but generally, I don't find this too beautiful. But, maybe, that's just my (horribly broken) taste. ;-) -- Heiko Wundram Product Application Development ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: OT: getting the protocol family of a file descriptor
Heiko Wundram (Beenic) [EMAIL PROTECTED] writes: Currently, you're basically required to do a getsockname to a struct sockaddr_storage and typecast that to the actual socket addres type based on the ss_family member (to be able to pass it to one of the *_ntop-functions, for example), but generally, I don't find this too beautiful. But, maybe, that's just my (horribly broken) taste. ;-) Uh, I'm pretty sure there's a function in the getaddr* family that will give you a string representation of any struct sockaddr. Actually, I'm absolutely sure of it: getnameinfo() with NI_NUMERICHOST|NI_NUMERICSERV will format the numerical address for you instead of looking it up in DNS. But what I really don't understand is this: you say you just want the address family so you'll know how to format the address. If you have the address, you already know the family. So what's the issue, really? DES -- Dag-Erling Smørgrav - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
On Thu, 31 Jan 2008 08:45:55 +0200 Adrian Penisoara [EMAIL PROTECTED] wrote: Side-topic, if you bear with me: if you were to choose again what to use as source revision control system (VCS) from today's offerings, what would you choose to maintain FreeBSD's sources or a side-off project tracking FreeBSD as base that would allow better teams cooperation and easy code merging between projects/branches ? Pretty much any post-CVS VCS will do that. But if you want a good merge facility, Perforce's are - well, after getting used to them, everything else feels like throwing your code against the wall and hoping the right parts stick. I talked to one of the git developers about a year ago, and they were thinking about adding a guided merge inspired by what Perforce does. For the moment I am thinking that the top contenders would be Bazaar and Mercurial but I would like to know other (developer) opinions. I last looked at distributed VCS systems about a year ago, and at the time liked Mercurial. The technology seems like it would be great for a project like FreeBSD. However, best practices for using them were still being worked out, and I'm not sure I'd want to commit a long-term project to one under those conditions. For a centralized VCS systems I've checked, perforce is the best of the post-CVS systems (and the only one that doesn't leave turds in the build tree). Subversion is a close second, but is still a little rough around the edges. Most notably, merge tracking is in the 1.5 beta builds, but not in the production code. mike -- Mike Meyer [EMAIL PROTECTED] http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: more on devstat (missing include)
Pietro Cerutti wrote: John Baldwin wrote: I think this might be best: Index: devstat.h === RCS file: /usr/cvs/src/lib/libdevstat/devstat.h,v retrieving revision 1.11 diff -u -r1.11 devstat.h --- devstat.h 22 Jul 2005 17:19:00 - 1.11 +++ devstat.h 31 Jan 2008 10:52:37 - @@ -32,6 +32,7 @@ #define _DEVSTAT_H #include sys/cdefs.h #include sys/devicestat.h +#include sys/resource.h #include kvm.h Good. Are you committing that or I better send in a PR? ;-) figured out myself.. Thanks! -- Pietro Cerutti PGP Public Key: http://gahr.ch/pgp signature.asc Description: OpenPGP digital signature
Re: [OT] Q: what would you choose for a VCS today
On Thu, 2008-01-31 at 11:02 -0500, Mike Meyer wrote: On Thu, 31 Jan 2008 08:45:55 +0200 Adrian Penisoara [EMAIL PROTECTED] wrote: Subversion is a close second, but is still a little rough around the edges. Most notably, merge tracking is in the 1.5 beta builds, but not in the production code. mike At $JOB, we moved to subversion from CVS about 2-3 years ago. We're still using subversion for everything, and use svnmerge.py [1] to manage development and release branches. It isn't ideal, as you lose information about the individual commits within a merged patchset, which makes it a minor pain to back out a specific commit from a merged patchset. Other features we're looking forward to is read-only slaves that post back commits to a global master site, which would greatly please our North American colleagues, save them from having to pull repos from over the pond. Tom [1] http://www.orcaware.com/svn/wiki/Svnmerge.py signature.asc Description: This is a digitally signed message part
Re: more on devstat (missing include)
On Thursday 31 January 2008 10:20:42 am Pietro Cerutti wrote:
John Baldwin wrote:
On Wednesday 30 January 2008 06:09:21 pm Pietro Cerutti wrote:
Hi hackers,
I think that i) sys/devicestat.h should include sys/resource.h or ii)
the man page of devstat(9) should mention that the user himself is
supposed to include sys/resource.h.
Without:
cat devstat.c
#include devstat.h
int main(void)
{
devstat_getnumdevs(NULL);
return (0);
}
gcc -ldevstat -o devstat devstat.c
In file included from devstat.c:1:
/usr/include/devstat.h:131: error: 'CPUSTATES' undeclared here (not in a
function)
Exit 1
I can provide a patch if someone could tell me which one of the proposed
solutions above is the preferred one.
I think this might be best:
Index: devstat.h
===
RCS file: /usr/cvs/src/lib/libdevstat/devstat.h,v
retrieving revision 1.11
diff -u -r1.11 devstat.h
--- devstat.h 22 Jul 2005 17:19:00 - 1.11
+++ devstat.h 31 Jan 2008 10:52:37 -
@@ -32,6 +32,7 @@
#define _DEVSTAT_H
#include sys/cdefs.h
#include sys/devicestat.h
+#include sys/resource.h
#include kvm.h
Good. Are you committing that or I better send in a PR?
I'll commit.
--
John Baldwin
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: OT: getting the protocol family of a file descriptor
Am Donnerstag, 31. Januar 2008 17:50:20 schrieb Dag-Erling Smørgrav:
Heiko Wundram (Beenic) [EMAIL PROTECTED] writes:
Currently, you're basically required to do a getsockname to a struct
sockaddr_storage and typecast that to the actual socket addres type
based on the ss_family member (to be able to pass it to one of the
*_ntop-functions, for example), but generally, I don't find this too
beautiful. But, maybe, that's just my (horribly broken) taste. ;-)
Uh, I'm pretty sure there's a function in the getaddr* family that will
give you a string representation of any struct sockaddr. Actually, I'm
absolutely sure of it: getnameinfo() with NI_NUMERICHOST|NI_NUMERICSERV
will format the numerical address for you instead of looking it up in
DNS.
But what I really don't understand is this: you say you just want the
address family so you'll know how to format the address. If you have
the address, you already know the family. So what's the issue, really?
Okay, formatting the address was just an example (and probably a bad one at
that), and is actually a negligible (diagnostic) part in the backend service
which gets passed a file-descriptor over a Unix Domain socket from other
daemons (actually, filters) running on the same machine.
What is the case here is that the server I'm building is basically a service
which works over any kind of SOCK_STREAM, but has to adapt its behaviour
slightly depending on the type of the socket being passed in. At the moment,
there are two different kinds of connections being handled by front-end
plugins (which basically accept on a listening socket, set up a certain
initial state and pass it to the backend): AF_INET(6) and AF_BLUETOOTH. The
latter won't work with the getaddr*-functions or getnameinfo() to format the
address, and also requires me to handle the connection slightly differently
in parts (because the information being served is adapted for RFCOMM [i.e.,
mobile] delivery).
The plugins which accept connections and pass them back aren't written by me,
and as such, I wanted to do at least some error checking on the passed in
socket (i.e., is SOCK_STREAM, has a family I support, isn't a listening
socket, etc.), and what I currently do is similar to the following (that's
just pseudo-code, beware of anything I got wrong from my head now):
struct sockaddr_storage addr;
socklen_t addrlen = sizeof(addr);
getsockname(fd,reinterpret_caststruct sockaddr*(addr),addrlen);
switch( addr-ss_family ) {
case AF_INET:
// ...
break;
case AF_INET6:
// Set up some stuff...
getpeername(fd,reinterpret_caststruct sockaddr*(addr),addrlen);
...inet_ntop(reinterpret_caststruct sockaddr*(addr));
break;
case AF_BLUETOOTH:
// Set up some other stuff...
getpeername(fd,reinterpret_caststruct sockaddr*(addr),addrlen);
bt_ntoa(reinterpret_caststruct
sockaddr_rfcomm*(addr)-rfcomm_bdaddr);
break;
default:
// We don't know this, ignore.
}
I personally don't find this especially beautiful, and generally, as there is
a getsockopt(SO_TYPE), I'd have thought the above should look (somewhat)
similar to the following:
int opt;
socklen_t optlen = sizeof(opt);
getsockopt(fd,SOL_SOCKET,SO_DOMAIN,opt,optlen);
switch( opt ) {
case PF_INET:
// ...
break;
case PF_INET6:
// Initialize inet connection.
initInet6();
break;
case PF_BLUETOOTH:
// Initialize bluetooth.
initBluetooth();
break;
default:
// Do something else.
}
where initBluetooth() is
struct sockaddr_rfcomm addr;
socklen_t addrlen = sizeof(addr);
getpeername(fd,reinterpret_caststruct sockaddr*(addr),addrlen);
bt_ntoa(...addr.rfcomm_bdaddr);
There's just one explicit type-cast in the latter (which makes it so much more
readable IMHO).
Anyway, as I said before, this is basically a style issue, and because a
socket is constructed with an explicit domain parameter I'd have thought that
there is some way to query that constructor argument (and explicitly _only_
that constructor argument), but apparently, nobody else feels the way I do,
which I can accept, don't worry. ;-)
--
Heiko Wundram
Product Application Development
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
OutBackDingo wrote: I'm having to use mercurial. I'm not really enjoying it. works ok for small projects. BSD is a bit big for it. doe work foe offline editing, but loses all your BSD history. probably SVK is the way to go from what I hear. Im using mercurial on full FreeBSD trees, curiosity makes me ask where do you the deficiency? Ive had no issues patching, branching, merging, transplanting, tracking vendor updates. The only issue i really had was a import of the full cvs tree so if I ask you to show me version 1.3 of ng_base.c and compare it to version 1.5, how do you do that? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Squeezing out some 70 bytes out of the boot2 loader
At 21:39 20/01/2008, you wrote: Hello I am trying to hack in some symlink support into the [sys/boot/i386/]boot2 bootloader (for my project [1]) and I seem to fall short of about 69 bytes: as -o boot2.o boot2.s ld -static -N --gc-sections -nostdlib -Ttext 0x2000 -o boot2.out/build/obj/build/src/sys/boot/i386/boot2/../btx/lib/crt0.o boot2.o sio.o objcopy -S -O binary boot2.out boot2.bin btxld -v -E 0x2000 -f bin -b /build/obj/build/src/sys/boot/i386/boot2/../btx/btx/btx -l boot2.ldr -o boot2.ld -P 1 boot2.bin kernel: ver=1.01 size=7b0 load=9000 entry=9010 map=16M pgctl=1:1 client: fmt=bin size=1581 text=0 data=0 bss=0 entry=0 output: fmt=bin size=1e45 text=114 data=1d31 org=0 entry=0 -69 bytes available *** Error code 1 What can I do to get room for about 70-100 bytes for these changes to make it into the bootloader ? [1] I'm trying to get support for /boot being mounted as a separate FS and as such I would need to have a self-pointing symlink (e.g. boot - . ) to easily mask the fact that the boot stuff is now right in the root of that FS. Fortunately the FORTH loader does support symlinks and I do not get problems with it. I know that I can use /boot.kernel as a workaround, but that is not too elegant. How big is boot2? You can compress it with an arithmetic coder, and only put the decoder plus the compressed boot2. There are very tiny arith coders, in assm, with 100-120 bytes of code (even less). One reason that life is complex is that it has a real part and an imaginary part -Andrew Koenig ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
At 8:45 AM +0200 1/31/08, Adrian Penisoara wrote: Hi, Side-topic, if you bear with me: if you were to choose again what to use as source revision control system (VCS) from today's offerings, what would you choose to maintain FreeBSD's sources or a side-off project tracking FreeBSD as base that would allow better teams cooperation and easy code merging between projects/branches ? You'll probably get a different answer from each person... :-) As for me, I'd go with subversion. I also believe 'git' might be a very interesting choice, but I haven't used it enough to know how well it works in practice. And I think that's the basic difficulty in trying to answer your question. Very few people have enough experience with all of the available VCS systems to do a comparison. I have worked a lot with RCS and CVS. I've done a little with perforce, but it is so different than CVS that I can't say that I gave it a fair chance. I just thought Oh boy, this is too weird!, and went on to some other project. I don't have enough time to take a real project, and try to make the same set of changes to multiple copies of the repository, to see which VCS *really* does a better job for everything which is needed. One of the guys I know swears that darcs is the best thing ever, and I can see how it would work well for some projects, but I can't imagine it working well for a project such as FreeBSD. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Memory allocation performance
Hi. While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: 0.14 0.05 132119/545292 ip_forward cycle 1 [12] 0.14 0.05 133127/545292 fxp_add_rfabuf [18] 0.27 0.10 266236/545292 ng_package_data [17] [9]14.1 0.56 0.21 545292 uma_zalloc_arg [9] 0.17 0.00 545292/1733401 critical_exit cycle 2 [98] 0.01 0.00 275941/679675 generic_bzero [68] 0.01 0.00 133127/133127 mb_ctor_pack [103] 0.15 0.06 133100/545266 mb_free_ext [22] 0.15 0.06 133121/545266 m_freem [15] 0.29 0.11 266236/545266 ng_free_item [16] [8]15.2 0.60 0.23 545266 uma_zfree_arg [8] 0.17 0.00 545266/1733401 critical_exit cycle 2 [98] 0.00 0.04 133100/133100 mb_dtor_pack [57] 0.00 0.00 134121/134121 mb_dtor_mbuf [111] I have already optimized all possible allocation calls and those that left are practically unavoidable. But even after this kgmon tells that 30% of CPU time consumed by memory management. So I have some questions: 1) Is it real situation or just profiler mistake? 2) If it is real then why UMA is so slow? I have tried to replace it in some places with preallocated TAILQ of required memory blocks protected by mutex and according to profiler I have got _much_ better results. Will it be a good practice to replace relatively small UMA zones with preallocated queue to avoid part of UMA calls? 3) I have seen that UMA does some kind of CPU cache affinity, but does it cost so much that it costs 30% CPU time on UP router? Thanks! -- Alexander Motin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Memory allocation performance
Alexander Motin пишет: While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: I have forgotten to tell that it was mostly GENERIC kernel just built without INVARIANTS, WITNESS and SMP but with 'profile 2'. -- Alexander Motin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
On Thu, Jan 31, 2008 at 08:45:55AM +0200, Adrian Penisoara wrote: Hi, Side-topic, if you bear with me: if you were to choose again what to use as source revision control system (VCS) from today's offerings, what would you choose to maintain FreeBSD's sources or a side-off project tracking FreeBSD as base that would allow better teams cooperation and easy code merging between projects/branches ? Finally a thread to vent about this topic :) I'd very much like to hear how others are doing this, please post, people, I'll read it all. Here's my take on it. I'm only talking about maintaining local changes, not what the FreeBSD project per se should use for change management, and also I don't really talk about working in a team but it's probably still relevant and might help: Don't _ever_ follow development(7) and try to maintain one or more custom branches inside a cvsup'd copy of the FreeBSD CVS repo. I've been down that road. It sucks. It really prevented me from getting any work done while waiting for hours on end for cvs to set a tag, do an update, a commit, let alone a merge. Every operation took ages and ages. With two branches, one based on RELENG_6 and CURRENT at the time, merging changes between the two was a major pain. CVS had so much overhead for me that using it made the whole point of doing version control fly out the window from, say, the 42nd floor. I've investigated quite a few options to maintain modifications to FreeBSD since, mainly to manage wake on lan patches (see http://stsp.name/wol/) but also local bug fixes I need for my system (e.g. enabling AIGLX does not lock up my 6.3 box so I can run compiz, see PR #114688). I'm quite serious about version control. In fact I'm a partial and (currently) paid committer for the subversion project, so I could even say that I'm involved in version control professionally. What I wanted instead of CVS sounds fairly simple to me: To maintain my local mods to FreeBSD I don't really care about the whole CVS history. I just need to be able to take a snapshot at some point, put it on a branch, and keep importing upstream changes incrementally to that branch from there on. So a vendor branch, but without any (or as little as possible) manual labour involved in updating it. I could not find any tool that does this properly among subversion, git, monotone and mercurial. That's not a big list, but I don't have time to try out version control systems all day. Also, proprietary VCS's were never considered (I also keep my freebsd kernels blob-free, call me a hippie or whatever if you want :P) Most tools seem to insist on trying to import the whole history of a CVS repository before they let you start doing any work in the newly converted repository. All conversion tools I've tried failed converting the FreeBSD repository. git-cvsimport fails after a few minutes because cvsps produces bad output when run on the FreeBSD repo. I reported this to the git developers and as a result they made git-cvsimport error out correctly, but did not fix the actual issue. The monotone built-in cvs converter segfaulted after running a whole day. The generic tailor VCS conversion tool failed as well -- I don't remember how, it errored out after running for a while. Even though I am subversion dev I did not try cvs2svn, because I wanted to take this as an opportunity to get my feet wet in another VCS. Mercurial failed to convert the repo, too, and there was no way of telling it not to try to import the whole history either. But its handbook describes interesting alternative approach to vendor branches: Patch queues. If you think you need a vendor branch, take a look at mercurial patch queues and consider if they might do the job just as well: Managing change with Mercurial Queues: http://hgbook.red-bean.com/hgbookch12.html#x16-26700012 Advanced uses of Mercurial Queues: http://hgbook.red-bean.com/hgbookch13.html#x17-30200013 I won't explain the details in this mail, as duplicating information from the handbook is a waste of time, but I'll give you my opinion: Patch queues are quite powerful, and even though you end up versioning diffs instead of whole files, the patch queue provides a nice enough abstraction that makes maintaining local changes as comfortable as maintaining a vendor branch. A big plus is that you do not need to take an extra step to generate diffs to send upstream, because you already have the diffs right in your .hg/patches directory. Conflict resolution works almost the same way as during a normal VCS's merge (whatever normal means in version control land :P), and as you get to incrementally make the patches in your queue apply again, you don't have to deal with a source tree full of all conflicts of a merge, but only with conflicts caused by a single patch at a time. Patch guards let you apply patches conditionally, this is where it gets interesting if you maintain changes for, say, RELENG_7 and CURRENT at the same time, and still
Re: Memory allocation performance
Alexander Motin wrote: Hi. While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: 0.14 0.05 132119/545292 ip_forward cycle 1 [12] 0.14 0.05 133127/545292 fxp_add_rfabuf [18] 0.27 0.10 266236/545292 ng_package_data [17] [9]14.1 0.56 0.21 545292 uma_zalloc_arg [9] 0.17 0.00 545292/1733401 critical_exit cycle 2 [98] 0.01 0.00 275941/679675 generic_bzero [68] 0.01 0.00 133127/133127 mb_ctor_pack [103] 0.15 0.06 133100/545266 mb_free_ext [22] 0.15 0.06 133121/545266 m_freem [15] 0.29 0.11 266236/545266 ng_free_item [16] [8]15.2 0.60 0.23 545266 uma_zfree_arg [8] 0.17 0.00 545266/1733401 critical_exit cycle 2 [98] 0.00 0.04 133100/133100 mb_dtor_pack [57] 0.00 0.00 134121/134121 mb_dtor_mbuf [111] I have already optimized all possible allocation calls and those that left are practically unavoidable. But even after this kgmon tells that 30% of CPU time consumed by memory management. So I have some questions: 1) Is it real situation or just profiler mistake? 2) If it is real then why UMA is so slow? I have tried to replace it in some places with preallocated TAILQ of required memory blocks protected by mutex and according to profiler I have got _much_ better results. Will it be a good practice to replace relatively small UMA zones with preallocated queue to avoid part of UMA calls? 3) I have seen that UMA does some kind of CPU cache affinity, but does it cost so much that it costs 30% CPU time on UP router? Make sure you have INVARIANTS disabled, it has a high performance cost in UMA. Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Memory allocation performance
Alexander Motin wrote: Hi. While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: 0.14 0.05 132119/545292 ip_forward cycle 1 [12] 0.14 0.05 133127/545292 fxp_add_rfabuf [18] 0.27 0.10 266236/545292 ng_package_data [17] [9]14.1 0.56 0.21 545292 uma_zalloc_arg [9] 0.17 0.00 545292/1733401 critical_exit cycle 2 [98] 0.01 0.00 275941/679675 generic_bzero [68] 0.01 0.00 133127/133127 mb_ctor_pack [103] 0.15 0.06 133100/545266 mb_free_ext [22] 0.15 0.06 133121/545266 m_freem [15] 0.29 0.11 266236/545266 ng_free_item [16] [8]15.2 0.60 0.23 545266 uma_zfree_arg [8] 0.17 0.00 545266/1733401 critical_exit cycle 2 [98] 0.00 0.04 133100/133100 mb_dtor_pack [57] 0.00 0.00 134121/134121 mb_dtor_mbuf [111] I have already optimized all possible allocation calls and those that left are practically unavoidable. But even after this kgmon tells that 30% of CPU time consumed by memory management. So I have some questions: 1) Is it real situation or just profiler mistake? 2) If it is real then why UMA is so slow? I have tried to replace it in some places with preallocated TAILQ of required memory blocks protected by mutex and according to profiler I have got _much_ better results. Will it be a good practice to replace relatively small UMA zones with preallocated queue to avoid part of UMA calls? 3) I have seen that UMA does some kind of CPU cache affinity, but does it cost so much that it costs 30% CPU time on UP router? given this information, I would add an 'item cache' in ng_base.c (hmm do I already have one?) Thanks! ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Memory allocation performance
Alexander Motin wrote: Alexander Motin пишет: While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: I have forgotten to tell that it was mostly GENERIC kernel just built without INVARIANTS, WITNESS and SMP but with 'profile 2'. What is 'profile 2'? Kris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
On 2008-01-31 21:37, Stefan Sperling [EMAIL PROTECTED] wrote:
I could not find any tool that does this properly among subversion,
git, monotone and mercurial. That's not a big list, but I don't have
time to try out version control systems all day. Also, proprietary
VCS's were never considered (I also keep my freebsd kernels blob-free,
call me a hippie or whatever if you want :P)
Most tools seem to insist on trying to import the whole history of a
CVS repository before they let you start doing any work in the newly
converted repository. All conversion tools I've tried failed converting
the FreeBSD repository.
Not really. You can keep 'importing' snapshots of the src tree from any
arbitrary CVS branch, if you are willing to wait until CVS checks out
the first copy of the snapshot.
This is how we 'resync' with the official doc/ tree changes in the Greek
translation team:
(a) We keep a Mercurial workspace which is read-only to everyone
else, except the importer.
(b) The importer checks outs doc/ snapshots and commits them as
'vendor code drops' in http://hg.hellug.gr/freebsd/doc/
(c) I pull changes from the 'import tree' into my own personal
workspace, and merge them with the latest translation effort text.
(d) Then the merged tree is pushed to a second 'workspace', 'branch'
or whatever you prefer calling it, at http://hg.hellug.gr/freebsd/doc-el/
The whole process of importing clean snapshots is automated in a shell
script, which I manually kick off at this point:
% #!/bin/sh
%
% cd /ws/doc/bsd
%
% # 1. Start from a clean slate for the next import
% rm -fr *
%
% # 2. Check out a clean copy of a partial doc/ tree.
% cvs -d /home/ncvs co -d . -l doc
% cvs -d /home/ncvs -qR up -APd * share \
% en_US.ISO8859-1 el_GR.ISO8859-7
% find . -type d -name CVS -exec rm -r {} +
%
% # 3. Update mercurial's idea of the current workspace state,
% # hg adding new files, and hg removing gone stuff.
% hg addremove
%
% # 4. Find out the $FreeBSD$ timestamp of the latest patch we are
% # about to commit. Note that this may be a bit silly, because it
% # won't correctly detect -kb files being added after the last
% # $FreeBSD$ id change. A better way would use -D to checkout from
% # CVS, so that a timestamp would be automatically known.
% timestamp=$( hg diff | grep '^+.*FreeBSD:' | \
% sed -e 's/.*,v //' | awk '{print $1,$2}' )
%
% # Commit everything to Mercurial.
% hg ci -u ncvs -d ${timestamp} + \
% -m Import FreeBSD doc/ snapshot at ${timestamp} +
That's not something I would like doing manually several times a day,
but it certainly isn't impossible.
Naturally, similar scripting can be installed for Subversion, Git,
Bazaar, or darcs if that's your personal preference.
Mercurial failed to convert the repo, too, and there was no way of
telling it not to try to import the whole history either.
Snapshot-based import sof FreeBSD code as `vendor imports' are really
*VERY* easy to script in Subversion, Mercurial, Git and Bazaar. Been
there, done that several times, and I can help you if you plan to do
something like this with any of the aforementioned VCSes.
So far, this setup hasn't failed me, and the speed is several orders of
magnitude higher than using CVS branches.
That's my impression too :)
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: [OT] Q: what would you choose for a VCS today
On 2008-02-01 02:00, Giorgos Keramidas [EMAIL PROTECTED] wrote: You can keep 'importing' snapshots of the src tree from any arbitrary CVS branch, if you are willing to wait until CVS checks out the first copy of the snapshot. This is how we 'resync' with the official doc/ tree changes in the Greek translation team: (a) We keep a Mercurial workspace which is read-only to everyone else, except the importer. (b) The importer checks outs doc/ snapshots and commits them as 'vendor code drops' in http://hg.hellug.gr/freebsd/doc/ (c) I pull changes from the 'import tree' into my own personal workspace, and merge them with the latest translation effort text. (d) Then the merged tree is pushed to a second 'workspace', 'branch' or whatever you prefer calling it, at http://hg.hellug.gr/freebsd/doc-el/ The whole process of importing clean snapshots is automated in a shell script, which I manually kick off at this point: An much improved snapshot import script is now finished (for some odd definition of `improved' I guess), even thought it is still a bit ugly for my taste. http://people.freebsd.org/~keramida/scripts/bsd-doc-import.ksh.txt I'd probably prefer Perl for some of the stuff done in ksh(1) there, but no time for that tonight, and it seems to work as a 'proof of concept' of importing partial checkouts from CVS to Hg without having to go through all the hoops of converting the *full* history. The cron job entry which runs this is: # Try to import a snapshot of the BSD doc/ tree once an hour. @hourly $HOME/bsd-doc-import.sh $HOME/hg/doc/bsd-import This is getting pretty off-topic for freebsd-hackers though, so it's probably time for me to shuttup and go do something useful :) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Memory allocation performance
Kris Kennaway пишет: Alexander Motin wrote: Alexander Motin пишет: While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: I have forgotten to tell that it was mostly GENERIC kernel just built without INVARIANTS, WITNESS and SMP but with 'profile 2'. What is 'profile 2'? I have thought it is high resolution profiling support. Isn't it? -- Alexander Motin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Memory allocation performance
Julian Elischer пишет: Alexander Motin wrote: Hi. While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: 0.14 0.05 132119/545292 ip_forward cycle 1 [12] 0.14 0.05 133127/545292 fxp_add_rfabuf [18] 0.27 0.10 266236/545292 ng_package_data [17] [9]14.1 0.56 0.21 545292 uma_zalloc_arg [9] 0.17 0.00 545292/1733401 critical_exit cycle 2 [98] 0.01 0.00 275941/679675 generic_bzero [68] 0.01 0.00 133127/133127 mb_ctor_pack [103] 0.15 0.06 133100/545266 mb_free_ext [22] 0.15 0.06 133121/545266 m_freem [15] 0.29 0.11 266236/545266 ng_free_item [16] [8]15.2 0.60 0.23 545266 uma_zfree_arg [8] 0.17 0.00 545266/1733401 critical_exit cycle 2 [98] 0.00 0.04 133100/133100 mb_dtor_pack [57] 0.00 0.00 134121/134121 mb_dtor_mbuf [111] I have already optimized all possible allocation calls and those that left are practically unavoidable. But even after this kgmon tells that 30% of CPU time consumed by memory management. So I have some questions: 1) Is it real situation or just profiler mistake? 2) If it is real then why UMA is so slow? I have tried to replace it in some places with preallocated TAILQ of required memory blocks protected by mutex and according to profiler I have got _much_ better results. Will it be a good practice to replace relatively small UMA zones with preallocated queue to avoid part of UMA calls? 3) I have seen that UMA does some kind of CPU cache affinity, but does it cost so much that it costs 30% CPU time on UP router? given this information, I would add an 'item cache' in ng_base.c (hmm do I already have one?) That was actually my second question. As there is only 512 items by default and they are small in size I can easily preallocate them all on boot. But is it a good way? Why UMA can't do just the same when I have created zone with specified element size and maximum number of objects? What is the principal difference? -- Alexander Motin ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Memory allocation performance
Alexander Motin wrote: Julian Elischer пишет: Alexander Motin wrote: Hi. While profiling netgraph operation on UP HEAD router I have found that huge amount of time it spent on memory allocation/deallocation: 0.14 0.05 132119/545292 ip_forward cycle 1 [12] 0.14 0.05 133127/545292 fxp_add_rfabuf [18] 0.27 0.10 266236/545292 ng_package_data [17] [9]14.1 0.56 0.21 545292 uma_zalloc_arg [9] 0.17 0.00 545292/1733401 critical_exit cycle 2 [98] 0.01 0.00 275941/679675 generic_bzero [68] 0.01 0.00 133127/133127 mb_ctor_pack [103] 0.15 0.06 133100/545266 mb_free_ext [22] 0.15 0.06 133121/545266 m_freem [15] 0.29 0.11 266236/545266 ng_free_item [16] [8]15.2 0.60 0.23 545266 uma_zfree_arg [8] 0.17 0.00 545266/1733401 critical_exit cycle 2 [98] 0.00 0.04 133100/133100 mb_dtor_pack [57] 0.00 0.00 134121/134121 mb_dtor_mbuf [111] I have already optimized all possible allocation calls and those that left are practically unavoidable. But even after this kgmon tells that 30% of CPU time consumed by memory management. So I have some questions: 1) Is it real situation or just profiler mistake? 2) If it is real then why UMA is so slow? I have tried to replace it in some places with preallocated TAILQ of required memory blocks protected by mutex and according to profiler I have got _much_ better results. Will it be a good practice to replace relatively small UMA zones with preallocated queue to avoid part of UMA calls? 3) I have seen that UMA does some kind of CPU cache affinity, but does it cost so much that it costs 30% CPU time on UP router? given this information, I would add an 'item cache' in ng_base.c (hmm do I already have one?) That was actually my second question. As there is only 512 items by default and they are small in size I can easily preallocate them all on boot. But is it a good way? Why UMA can't do just the same when I have created zone with specified element size and maximum number of objects? What is the principal difference? who knows what uma does.. but if you do it yourself you know what the overhead is.. :-) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
