On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Tue, Jul 31, 2012 at 4:14 PM, Attilio Rao atti...@freebsd.org wrote: You don't want to work cooperatively. Why is it that mbuf's refactoring consultation is being held in internal, private, committers-and-invite-only-restricted meeting at BSDCan ? Why is it that so much review and discussion on changes are held privately ? - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 12:40 PM, Attilio Rao atti...@freebsd.org wrote: On Wed, Aug 1, 2012 at 5:32 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Tue, Jul 31, 2012 at 4:14 PM, Attilio Rao atti...@freebsd.org wrote: You don't want to work cooperatively. Why is it that mbuf's refactoring consultation is being held in internal, private, committers-and-invite-only-restricted meeting at BSDCan ? Why is it that so much review and discussion on changes are held privately ? Arnaud, belive me, to date I don't recall a single major technical decision that has been settled exclusively in private (not subjected to peer review) and in particular in person (e-mail help you focus on a lot of different details that you may not have under control when talking to people, etc). Whose call is it to declare something worth public discussion ? No one. Every time I see a Suggested by:, Submitted by:, Reported by:, and especially Approved by:, there should to be a public reference of the mentioned communication. Sometimes it is useful that a limited number of developers is involved in initial brainstorming of some works, Never. but after this period constructive people usually ask for peer review publishing their plans on the mailing lists or other media. Again, never. By doing so, you merely put the community in a situation where, well, We, committers, have come with this, you can either accept or STFU, but no major changes will be made because we decided so. The callout-ng conference at BSDCan was just beautiful, it was basically: Speaker: we will do this Audience: how about this situation ? What you will do will not work. Speaker: thank you for listening, end of the conference It was beautiful to witness. If you don't see any public further discussion this may be meaning: a) the BSDCan meetings have been fruitless and there is no precise plan/roadmap/etc. so not only you make it private, but it shamelessly failed... b) there is still not consensus on details Then the discussion should stop, public records are kept for reference in the future. There is no problem with this. and you can always publically asked on what was decided and what not. Just send a mail to interested recipients and CC any FreeBSD mailing list. This is not the way openness should be about. - Arnaud Attilio -- Peace can only be achieved by understanding - A. Einstein ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 2:18 PM, Attilio Rao atti...@freebsd.org wrote: On 8/1/12, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Wed, Aug 1, 2012 at 12:40 PM, Attilio Rao atti...@freebsd.org wrote: On Wed, Aug 1, 2012 at 5:32 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Tue, Jul 31, 2012 at 4:14 PM, Attilio Rao atti...@freebsd.org wrote: You don't want to work cooperatively. Why is it that mbuf's refactoring consultation is being held in internal, private, committers-and-invite-only-restricted meeting at BSDCan ? Why is it that so much review and discussion on changes are held privately ? Arnaud, belive me, to date I don't recall a single major technical decision that has been settled exclusively in private (not subjected to peer review) and in particular in person (e-mail help you focus on a lot of different details that you may not have under control when talking to people, etc). Whose call is it to declare something worth public discussion ? No one. Every time I see a Suggested by:, Submitted by:, Reported by:, and especially Approved by:, there should to be a public reference of the mentioned communication. Not necessarilly. Every developer must ensure to produce a quality work, with definition of quality being discretional. Some people fail this expectation, while others do very good. As a general rule, some people send patches to experts on the matter and they just trust their judgment, others also full-fill testing cycles by thirdy part people, others again ask for public reviews. Often this process is adapted based on the dimension of the work and the number of architectural changes it introduces. As a personal matter, for a big architectural change things I would *personally* do are: - Prepare a master-plan with experts of the matter - Post a plan (after having achived consensus) on the public mailing list for further discussions - Adjust the plan based on public feedbacks (if necessary) - Implement the plan - Ask the experts if they have objections to the implementation - Ask testers to perform some stress-testing - Ask testers to perform benchmark (if I find people interested in that) - Send out to the public mailing list for public review - Integrate suggestions - Ask testers to stress-test again - Commit I think this model in general works fairly well, I agree. but people may have different ideas on that, meaning that people may want to not involve thirdy part for testing or review. This is going to be risky and lower the quality of their work but it is their call. Sometimes it is useful that a limited number of developers is involved in initial brainstorming of some works, Never. but after this period constructive people usually ask for peer review publishing their plans on the mailing lists or other media. Again, never. By doing so, you merely put the community in a situation where, well, We, committers, have come with this, you can either accept or STFU, but no major changes will be made because we decided so. You are forgetting one specific detail: you can always review a work *after* it entered the tree. This is something you would never do, but sometimes, when poor quality code is committed there is nothing else you can do than just raise your concern after it is in. Unfortunately, not. First, the developer will certainly have moved on after the commit, API may have been changed tree-wide, etc. Then, time is likely to have passed between the time you figure potential regression or bugs, which makes the first point even more problematic. Finally, if my point of view is being ignored *before* it goes to the tree, do you really expect it will be considered *after* ? From my external point of view, committers not only have the possibility, but *do* commit mess in the tree without non-committers could say or do anything, just as well as committers being able to arbitrarily close PR even if the original reporter disagree. The callout-ng conference at BSDCan was just beautiful, it was basically: Speaker: we will do this Audience: how about this situation ? What you will do will not work. Speaker: thank you for listening, end of the conference It was beautiful to witness. Not sure if you realized but I was what you mention Audience. I think you are referring to a specific case where a quick heads-up on a summer of code project has been presented, you cannot really believe all the technical discussion around FreeBSD evolve this way. indeed, but that was still the case back then. If you don't see any public further discussion this may be meaning: a) the BSDCan meetings have been fruitless and there is no precise plan/roadmap/etc. so not only you make it private, but it shamelessly failed... And so? I think you have a wrong point of view of what is shamelessly... I'm working on the same project since 6 months, i thought I could finish it in 1 but then I understood that in order
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 4:06 PM, Adrian Chadd adr...@freebsd.org wrote: Any interested party is very welcome to approach a developer and get added to the developer summits. Plenty of the people at the most recent developer summit weren't @freebsd.org committers - we had plenty of representation from companies using FreeBSD. If you want to participate, just ask a friendly developer who is going to the developer summit to sponsor you in going. You're pleasant in person, so I'd have no problem sponsoring you if I am going to an event. :) I have a very deep, quasi-philosophical, trouble/problem with that whole idea of sponsor-requirement to attend a such meeting. There is just something which does not feel right about it. From my point of view, this is a matter of common sense, focus is gonna be very narrow and deeply technical. Attendee should go there only if they think they will give positive feedback. As for myself, I would not attend a developer meeting on the fiber-channel over infiniband optimization, but would attend a developer meeting on next-generation mbuf. Now, maybe I'll just push the door of some developer meeting I'd be interested in during next BSDCan, and see what happen :-) The outcome might be interesting to study in a social interaction, prisoner dilemma related, point-of-view. - Arnaud And/or, work with warner to get improvements into the tree and someone will sponsor a commit bit for you. Perhaps we as developers should more openly publish the results of developer summits. But as I said, they're not closed - they're just invite only for non-developers. We're not going to exclude anyone from coming unless they really ARE going to just sit there and troll. You're motivated, you're enthusiastic and you want to see things change for the better. You're also not confrontational in person. I have no problem with you coming along. Adrian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: On cooperative work [Was: Re: newbus' ivar's limitation..]
Hi, On Wed, Aug 1, 2012 at 7:28 PM, Warner Losh i...@bsdimp.com wrote: On Aug 1, 2012, at 3:39 PM, Arnaud Lacombe wrote: Hi, On Wed, Aug 1, 2012 at 4:06 PM, Adrian Chadd adr...@freebsd.org wrote: Any interested party is very welcome to approach a developer and get added to the developer summits. Plenty of the people at the most recent developer summit weren't @freebsd.org committers - we had plenty of representation from companies using FreeBSD. If you want to participate, just ask a friendly developer who is going to the developer summit to sponsor you in going. You're pleasant in person, so I'd have no problem sponsoring you if I am going to an event. :) I have a very deep, quasi-philosophical, trouble/problem with that whole idea of sponsor-requirement to attend a such meeting. There is just something which does not feel right about it. From my point of view, this is a matter of common sense, focus is gonna be very narrow and deeply technical. Attendee should go there only if they think they will give positive feedback. As for myself, I would not attend a developer meeting on the fiber-channel over infiniband optimization, but would attend a developer meeting on next-generation mbuf. Now, maybe I'll just push the door of some developer meeting I'd be interested in during next BSDCan, and see what happen :-) The outcome might be interesting to study in a social interaction, prisoner dilemma related, point-of-view. Given how ridiculously easy it is to get a proper invite, there's not need to be a jerk just to prove an obscure philosophical point about attendance. There's plenty of time to do that over the technical points being discussed. Let me explain my thoughts: I do not recognize the committers legitimacy to give such invite, and to some extend, I do not recognize committers self-given legitimacy altogether. This do not mean I'd praised a structure-less project; quite the opposite actually. Starting from that, I will certainly not defer to anybody to request such invite or commit bit. Feel free to kick me out of the meeting room if you want to; I would have proved my point. Now, if invites are so easy to get, just get rid of it. It's a worthless, cumbersome item. - Arnaud ps: please, do not get me wrong, I would apply this policy to anybody who propose to help. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 30, 2012 at 11:51 PM, Warner Losh i...@bsdimp.com wrote: [...] We lack that right now, which is why you're trying to shoe-horn the FDT connections into a newbus world and complaining that everything sucks because it is a poor fit. I'd suggest that different mechanisms are necessary. I'm not trying anything, or at least no longer. I do not see the point working on that when I can not get trivial patches in, especially those fixing poorly maintained drivers, whose issues _do_ affect people. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Tue, Jul 31, 2012 at 12:27 PM, Warner Losh i...@bsdimp.com wrote: On Jul 31, 2012, at 9:20 AM, Arnaud Lacombe wrote: Hi, On Mon, Jul 30, 2012 at 11:51 PM, Warner Losh i...@bsdimp.com wrote: [...] We lack that right now, which is why you're trying to shoe-horn the FDT connections into a newbus world and complaining that everything sucks because it is a poor fit. I'd suggest that different mechanisms are necessary. I'm not trying anything, or at least no longer. I do not see the point working on that when I can not get trivial patches in, especially those fixing poorly maintained drivers, whose issues _do_ affect people. Hey Arnaud, sorry to be a little harsh, but maybe if you shouted less and cooperated more, people would be more willing to work with you. I tried to be Mr Nice Guy in the past, all I got in return was being ignored. Lists are full of unanswered problem because people do not want to insist getting an answer. Now, believe me on one point, if you are a driver or subsystem author, might I have an issue with your work, I *will* be a recurring pain in your butt to get the issue fixed, or to get in what I do believe, with the limited set of knowledge and resources to my disposal[0], to be a correct fix for the issue, at the time. If you are condescending, arrogant, or advocates of the status-quo, as have been committers in the past, I will return you favor. Let face it, FreeBSD is not the most outstanding OS out there (despite obvious capabilities), and I would not be too proud of its state. All that to say that asking politely does not work in the arbitrary FreeBSD realm, where the power to serve, is today nothing more that a relic. - Arnaud ps: note that I am ready to be wrong and to be publicly proven so, as it's been the case recently. [0]: which certainly not mean I am wrong, very far from that. Having said that, I'd be happy to help fix this problem. I've not seen the patches you're talking about, I'd be happy to take a look and try to get them in, as appropriate. Warner ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi,
On Mon, Jul 30, 2012 at 5:06 PM, John Baldwin j...@freebsd.org wrote:
On Tuesday, July 17, 2012 2:03:14 am Arnaud Lacombe wrote:
Hi,
On Fri, Jul 13, 2012 at 1:56 PM, Arnaud Lacombe lacom...@gmail.com wrote:
Hi,
On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh i...@bsdimp.com wrote:
[..]
Honestly, though, I think you'll be more pissed when you find out that
the N:1 interface that you want is being done in the wrong domain. But I've
been wrong before and look forward to seeing your replacement.
I will just pass function pointers for now, if things should be done
dirty, let's be explicit about it.
Now, the hinted device attachment did work quite smoothly, however, I
would have a few suggestion:
1) add a call to bus_enumerate_hinted_children() before the call
DEVICE_IDENTIFY() call in bus_generic_driver_added()
this is required to be able to support dynamic loading and attachment
of hinted children.
I'm not sure this is a feature we want to support (to date hinted children
have only been created at boot time).
2) have a generic bus_hinted_child method which would just add a new
child to the bus.
Possibly, but hinted children are intentionally opt-in and not enabled
by default.
3) have bus_enumerate_hinted_children() and bus_generic_attach()
always ran on device attachment.
There is current +100 explicit call to bus_generic_attach() in the
sys/dev/ tree. This should be done always and implicitly.
No. One of the problems is that different busses want to do it at
different times. It is usually done last, but some buses may want to
do additional work after the bus_generic_attach().
4) have bus_generic_detach() always ran prior to device detachment
Similar.
5) have the bus_generic_* method be the default of their respective
method
No. However, what would be a good idea (and one thing I've had on my
list), is to create a bus_generic driver that uses the bus_generic_*
methods for all it's methods and let bus drivers inherit from that to
get the generic methods if they are appropriate. If you do this, I
would also add a second bus_generic_rl that inherits from bus_generic
but uses the generic resource list methods for resources.
6) have device_delete_child() called upon device detachment.
No. This is for a bus to decide. This would be horrifically wrong
for things like kldunloading a PCI device driver. Just because you
unload a driver (so that it detaches from devices) does not mean those
physical devices have gone away.
As a rule of thumb, when a kld is unloaded there should not be any
remains of anything built previously. Without device_delete_child() or
proper singleton implementation, multiple load/unload sequence of bus
will attempt to attach multiple version of a child, even if the single
child was added prior to the bus_generic_attach() call.
Hinted devices should perhaps be removed, yes. However, what other drivers
often do is use a singleton approach instead (despite your claim that they
don't).
FreeBSD's newbus device framework already sucks (as in too
static/inflexible), making it sucks even more (as in more
static/more inflexible) might not be the wisest approach... This is
no longer the 90'. The good old enumerating-buses, tree-based, model
is to be relegated to a corner-case of the computer world with
profusion of embedded, non-enumerating, highly integrated, highly
interconnected, highly custom SoCs.
I am yet to see a robust approach to the various problem I submitted.
For example:
static void
ipmi_isa_identify(driver_t *driver, device_t parent)
{
struct ipmi_get_info info;
uint32_t devid;
if (ipmi_smbios_identify(info) info.iface_type != SSIF_MODE
device_find_child(parent, ipmi, -1) == NULL) {
...
BUS_ADD_CHILD(parent, 0, ipmi, -1);
}
}
duplicated code doing the exact same abstract, hardcoded, function,
all over the tree, absolutely unacceptable, if not a blatant proof of
failure of what should be made generic, if not fully dynamic.
- Arnaud
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: libkern version of inet_ntoa_r
Hi,
On Sun, Jul 29, 2012 at 3:19 PM, Luigi Rizzo ri...@iet.unipi.it wrote:
Remapping f(a) into f(a, b) requires both a macro
and a wrapping function, something like this
T __f(T1 a, T2 b) { return f(a, b); }
#define f(a) __f(a, b)
This can be done way more easily:
void fn(int a, int b)
{
printf(%d %d\n, a, b);
}
#define fn(x) ({ fn(x, 42); })
int main(int argc, char **argv)
{
fn(0);
return 0;
}
works just fine.
but i am not so interested in participating to the IOCCC :)
maybe you should ;-)
- Arnaud
ps: this construct is used all over the Linux kernel compatibility libraries.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: libkern version of inet_ntoa_r
Hi,
On Sun, Jul 29, 2012 at 4:30 PM, Lev Serebryakov l...@freebsd.org wrote:
Hello, Luigi.
You wrote 30 июля 2012 г., 0:47:21:
#define fn(x) ({ fn(x, 42); })
LR nice trick, one always learns something on these lists...
LR now i wonder how it works with MSVC (windows being one of the
LR other platforms where i need to build the ipfw+dummynet code...)
It looks very gcc-ish.
could you back your point with a technical argument, please ? This
sounds rather FUD'ish so far.
The ({ ... }) is nothing more than a primary-expression enclosing a
compound-statement; this part is not specifically necessary. As for
the expansion, I would assume the following part of iso9899:1999
applies:
6.10.3 Macro replacement
10
A preprocessing directive of the form
# define identifier lparen identifier-listopt ) replacement-list new-line
# define identifier lparen ... ) replacement-list new-line
# define identifier lparen identifier-list , ... ) replacement-list new-line
defines a function-like macro with arguments, similar syntactically to
a function call. [...] Each subsequent instance of the function-like
macro name followed by a ( as the next preprocessing token introduces
the sequence of preprocessing tokens that is replaced by the
replacement list in the definition (an invocation of the macro). The
replaced sequence of preprocessing tokens is terminated by the
matching ) preprocessing token, [...]
Note that the standard say Each subsequent, no All, so
fn(1, 2);
#define fn(a) fn(a, 2)
fn(1);
would produces:
fn(1, 2);
fn(1, 2);
I do not see any gcc'ism here.
Now, I might be wrong, and would enjoy being proven so.
Thanks,
- Arnaud
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: panic: _mtx_lock_sleep: recursed on non-recursive mutex em0 @ /usr/src/sys/dev/e1000/if_lem.c:881
Hi, On Fri, Jul 27, 2012 at 4:31 PM, Adrian Chadd adr...@freebsd.org wrote: It looks like a case of lock held during call up the stack. This is bad for so many reasons. It also makes writing correctly locked drivers a pain in the ass as the moment you unlock the driver before calling ether_input() / ieee80211_input(), you allow things to change state. So no, although you shouldn't use long-held locks to protect things, apparently this is all the rage. iwn(4) does this. ath(4) does not. I'm having a right painful time trying to fine-grain lock things. I'm at the point where I'm about to not care, rip out all the locks and replace with a single ATH_LOCK(). How would a single ATH_LOCK() helps here ? AFAICS, the panic seem to be a classical fallout from direct dispatch where you can re-enter the driver from the driver itself through the network stack. - Arnaud Adrian On 27 July 2012 11:47, Dimitry Andric d...@freebsd.org wrote: On 2012-07-26 17:46, David Wolfskill wrote: This is at r238795; cut/paste of backtrace: KDB: enter: panic [ thread pid 12 tid 100026 ] Stopped at kdb_enter+0x3d: movl$0,kdb_why db bt Tracing pid 12 tid 100026 td 0xc6755000 kdb_enter(c0f93c5f,c0f93c5f,c0f91e21,f08398f0,c1825c80,...) at kdb_enter+0x3d panic(c0f91e21,c66739a0,c0f20db8,371,c6755000,...) at panic+0x1c4 _mtx_lock_sleep(c68b8560,c6755000,c0f20db8,c0f20db8,371,...) at _mtx_lock_sleep+0x35e _mtx_lock_flags(c68b8560,0,c0f20db8,371,0,...) at _mtx_lock_flags+0xdb lem_start(c68ba000,0,c0fa806c,d20,2a,...) at lem_start+0x33 if_transmit(c68ba000,c93d9000,6,c6755000,c65da588,...) at if_transmit+0x129 ether_output(c68ba000,c93d9000,f0839aa4,0,0,...) at ether_output+0x5df arpintr(c93d9000,8,c0f50314,153,0,...) at arpintr+0x108c netisr_dispatch_src(7,0,c93d9000,c93d9000,c68ba000,c93d0806) at netisr_dispatch_src+0xa7 netisr_dispatch(7,c93d9000,c93d9000,10,3,...) at netisr_dispatch+0x20 ether_demux(c68ba000,c93d9000,3,0,3,...) at ether_demux+0x133 ether_nh_input(c93d9000,c8f012c8,644d6000,c9492d00,0,...) at ether_nh_input+0x329 netisr_dispatch_src(9,0,c93d9000,e2e,2,1) at netisr_dispatch_src+0xa7 netisr_dispatch(9,c93d9000) at netisr_dispatch+0x20 ether_input(c68ba000,c93d9000,c0f20db8,e2e,c11454c0,...) at ether_input+0x21 lem_intr(c68b6000,8,c0f8e00d,561,0,...) at lem_intr+0x567 intr_event_execute_handlers(c11454c0,c6626200,c0f8e00d,561,c6755000,...) at intr_event_execute_handlers+0xc5 ithread_loop(c6627730,f0839d08,c0f8dd64,3db,0,...) at ithread_loop+0xe2 fork_exit(c0a2dcb0,c6627730,f0839d08) at fork_exit+0x7c fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xf0839d40, ebp = 0 --- db show locks exclusive sleep mutex em0 (EM TX Lock) r = 0 (0xc68b8560) locked @ /usr/src/sys/dev/e1000/if_lem.c:1324 exclusive sleep mutex em0 (EM Core Lock) r = 0 (0xc68b854c) locked @ /usr/src/sys/dev/e1000/if_lem.c:1302 db I need to head off to a meeting; I can poke at the machine a bit more in a couple of hours or so. I get the same panic and backtrace here, while running as a VMware guest. At least, as soon something actually happens with the network. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: panic: _mtx_lock_sleep: recursed on non-recursive mutex em0 @ /usr/src/sys/dev/e1000/if_lem.c:881
Hi, On Sat, Jul 28, 2012 at 4:04 PM, Adrian Chadd adr...@freebsd.org wrote: On 28 July 2012 12:09, Arnaud Lacombe lacom...@gmail.com wrote: How would a single ATH_LOCK() helps here ? AFAICS, the panic seem to be a classical fallout from direct dispatch where you can re-enter the driver from the driver itself through the network stack. Take a look at iwn. It has a single lock - IWN_LOCK() - which it releases before punting the frame up the stack. oh, I see. So, what happens in lem(4) is that we enter the stack with RX unlocked, but TX and CORE are still locked. The whole locking of lem(4) seems rather inconsistent. Through DEVICE_POLLING, lem_rxeof() is called with TX and CORE unlocked. Through EN_LEGACY_IRQ it is called with TX and CORE locked, and from MSI interrupt handler, is is caller with TX unlocked (CORE assumed to be unlock). I'd assume that lem(4) is just poorly maintained[0]... I would make a huge shot in the dark by proposing the completely untested attached patch :/ - Arnaud [0]: like code claiming support for Intel 82574 when this chipset *cannot* be used by lem(4), as there is no E1000_DEV_ID_82574* entries in `lem_vendor_info_array'. if_lem.c.diff Description: Binary data ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: libkern version of inet_ntoa_r
Hi,
On Sat, Jul 28, 2012 at 6:14 PM, Bjoern A. Zeeb
bzeeb-li...@lists.zabbadoz.net wrote:
On Wed, 25 Jul 2012, Luigi Rizzo wrote:
During some ipfw/dummynet cleanup i noticed that the libkern version of
inet_ntoa_r() is missing the buffer size argument that is present in
the libc counterpart.
Any objection if i fix it ?
And why exactly would you need it? What does libc do with it? Render
partial IPv4 addresses?
Mitigate possibilities of memory corruption ? At the very least, allow
the following:
{
char tmp[sizeof 255.255.255.255];
KASSERT(size = (sizeof tmp));
[...]
}
to be enforced... but hey, who gives a damn about consistently doing
things and enforcing code assumptions ? ;-)
- Arnaud
--
Bjoern A. Zeeb You have to have visions!
Stop bit received. Insert coin for new address family.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: libkern version of inet_ntoa_r
Hi,
On Sat, Jul 28, 2012 at 6:44 PM, Bjoern A. Zeeb
bzeeb-li...@lists.zabbadoz.net wrote:
On Sat, 28 Jul 2012, Arnaud Lacombe wrote:
Hi,
On Sat, Jul 28, 2012 at 6:14 PM, Bjoern A. Zeeb
bzeeb-li...@lists.zabbadoz.net wrote:
On Wed, 25 Jul 2012, Luigi Rizzo wrote:
During some ipfw/dummynet cleanup i noticed that the libkern version of
inet_ntoa_r() is missing the buffer size argument that is present in
the libc counterpart.
Any objection if i fix it ?
And why exactly would you need it? What does libc do with it? Render
partial IPv4 addresses?
Mitigate possibilities of memory corruption ? At the very least, allow
the following:
{
char tmp[sizeof 255.255.255.255];
char tmp[INET_ADDRSTRLEN];
Should I mention that this is taken directly from
`lib/libc/inet/inet_ntop.c' ? you may want to fix it, as you have been
blessed with a commit bit.
KASSERT(size = (sizeof tmp));
This would need to go into the called library function and cannot.
So that gives you what extra checking exactly? That the programmer got
the sizeof right rather than the buffer size? You pushed some more on the
stack or reused an register
That is funny. I was told that 2 always unused extra argument all
along the mutex API could not change anything, and now I am told the
opposite for 1 argument. There is no point trading the cost of a
register for overall runtime correctness. This is a string
manipulation function, it must be doing some kind of size check.
for something that is supposed to be at a minial fixed length
supposed to be ? you do not seem to be really sure to know how
inet_ntoa_r() is gonna be used, nor have you any way, by your words,
enforce it in any way.
(nothing else lower allowed and will ever result
in anything but misbehaviour) no matter what.
I'd be more than happy to welcome you tracking down memory corruption
based misbehaviors, but I prefer to detect it before, than attempt to
catch it after, it happens.
It's not like it's
inet_pton which can take totally different sizes.
that's nothing but an implementation details.
Which again leaves me with the question - why does libc have it?
It is passed down to strlcpy(). You could have found this out by
yourself in a minute or so... But again, implementation details, might
they be incomplete, are irrelevant.
- Arnaud
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: libkern version of inet_ntoa_r
Hi, On Sat, Jul 28, 2012 at 6:44 PM, Bjoern A. Zeeb bzeeb-li...@lists.zabbadoz.net wrote: Which again leaves me with the question - why does libc have it? as for the semantic, theoretical, why, I would refer you to the POSIX's comity, as inet_ntop() is part of it. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: RFC: libkern version of inet_ntoa_r
Hi,
On Sat, Jul 28, 2012 at 7:46 PM, Arnaud Lacombe lacom...@gmail.com wrote:
Hi,
On Sat, Jul 28, 2012 at 6:44 PM, Bjoern A. Zeeb
bzeeb-li...@lists.zabbadoz.net wrote:
Which again leaves me with the question - why does libc have it?
as for the semantic, theoretical, why, I would refer you to the
POSIX's comity, as inet_ntop() is part of it.
actually, it is slightly more complicated than that. POSIX has
inet_ntoa(), inet_ntop() and no inet_ntoa_r(). libc's inet_ntoa{,_r}()
are implemented on top inet_ntop(), which *does* fail if the provided
buffer is not large enough, and set errno to ENOSPC. However,
inet_ntoa{,_r}() do not propagate inet_ntop() failure.
As for the userland version of inet_ntoa{,_r}(), I would change it to
at least stop ignoring inet_ntop() return value, add an assertion on
its success, drop this awful `strcpy(ret, [inet_ntoa error]);' and
use the proper size in inet_ntoa() definition. As for the kernel
version... it is a lost cause to argue one way or another...
- Arnaud
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Fri, Jul 13, 2012 at 1:56 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh i...@bsdimp.com wrote: [..] Honestly, though, I think you'll be more pissed when you find out that the N:1 interface that you want is being done in the wrong domain. But I've been wrong before and look forward to seeing your replacement. I will just pass function pointers for now, if things should be done dirty, let's be explicit about it. Now, the hinted device attachment did work quite smoothly, however, I would have a few suggestion: 1) add a call to bus_enumerate_hinted_children() before the call DEVICE_IDENTIFY() call in bus_generic_driver_added() this is required to be able to support dynamic loading and attachment of hinted children. 2) have a generic bus_hinted_child method which would just add a new child to the bus. 3) have bus_enumerate_hinted_children() and bus_generic_attach() always ran on device attachment. There is current +100 explicit call to bus_generic_attach() in the sys/dev/ tree. This should be done always and implicitly. 4) have bus_generic_detach() always ran prior to device detachment If not already the case. There is still the same +100 direct call to bus_generic_detach is the tree. 5) have the bus_generic_* method be the default of their respective method 6) have device_delete_child() called upon device detachment. As a rule of thumb, when a kld is unloaded there should not be any remains of anything built previously. Without device_delete_child() or proper singleton implementation, multiple load/unload sequence of bus will attempt to attach multiple version of a child, even if the single child was added prior to the bus_generic_attach() call. Also, as a rule of thumb, if the same logic is implemented in more than a few buses, it should be made generic and implicit. I am lazy, I hate doing the same things over and over, not to say it raised the likelihood of bugs' introduction... could I at least get some feedback on the proposals above ? Thanks, - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh i...@bsdimp.com wrote: [..] Honestly, though, I think you'll be more pissed when you find out that the N:1 interface that you want is being done in the wrong domain. But I've been wrong before and look forward to seeing your replacement. I will just pass function pointers for now, if things should be done dirty, let's be explicit about it. Now, the hinted device attachment did work quite smoothly, however, I would have a few suggestion: 1) add a call to bus_enumerate_hinted_children() before the call DEVICE_IDENTIFY() call in bus_generic_driver_added() this is required to be able to support dynamic loading and attachment of hinted children. 2) have a generic bus_hinted_child method which would just add a new child to the bus. 3) have bus_enumerate_hinted_children() and bus_generic_attach() always ran on device attachment. There is current +100 explicit call to bus_generic_attach() in the sys/dev/ tree. This should be done always and implicitly. 4) have bus_generic_detach() always ran prior to device detachment If not already the case. There is still the same +100 direct call to bus_generic_detach is the tree. 5) have the bus_generic_* method be the default of their respective method 6) have device_delete_child() called upon device detachment. As a rule of thumb, when a kld is unloaded there should not be any remains of anything built previously. Without device_delete_child() or proper singleton implementation, multiple load/unload sequence of bus will attempt to attach multiple version of a child, even if the single child was added prior to the bus_generic_attach() call. Also, as a rule of thumb, if the same logic is implemented in more than a few buses, it should be made generic and implicit. I am lazy, I hate doing the same things over and over, not to say it raised the likelihood of bugs' introduction... - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Thu, Jul 12, 2012 at 1:20 AM, Warner Losh i...@bsdimp.com wrote: I'm sorry you feel that way. Honestly, though, I think you'll be more pissed when you find out that the N:1 interface that you want is being done in the wrong domain. But I've been wrong before and look forward to seeing your replacement. acpi_pcib_acpi.c, btw, implements both PCIB interfaces and ACPI interfaces. Does it ? From the definition of `acpi_pcib_acpi_methods', I can only see a single pcib(4) interface being exported. Moreover, I do not seem to be able to find any clue that would led any ACPI devices to attach on acpi_pcib_acpi(4), neither to find how could acpi_get_flags() ends up in acpi_pcib_read_ivar() ? Thks, - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 9, 2012 at 1:17 AM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Mon, Jul 9, 2012 at 12:37 AM, Warner Losh i...@bsdimp.com wrote: On Jul 8, 2012, at 9:46 PM, Arnaud Lacombe wrote: On Sun, Jul 8, 2012 at 11:31 PM, Warner Losh i...@bsdimp.com wrote: On Jul 8, 2012, at 9:26 PM, Arnaud Lacombe wrote: Hi, On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh i...@bsdimp.com wrote: On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote: Ok, yet another Newbus' limitation. Assuming a device exports more than one interface, and one of its child has need to use more than one interface, each interfaces cannot register, concurrently, its own ivar. While I try to always have a single child per interface/resource, I need to keep some compatibility with the old way of doing thing (POLA wrt. drivers I cannot/will not convert and userland). So, it would have been nice if ivar had been per-interface, not global and unique to one device. There's one pointer for the ivars. The bus code gets to determine what the ivar looks like, because the interface is totally private to the bus. So long as it returns the right thing for any key that's presented, it doesn't matter quite how things are done. So I'm not sure I understand what you are saying here. dev0 implements two interfaces: A and B. dev1, child of dev0, needs to use both interfaces. There is no generic way for dev0 to export independent ivars for both interface. For now, I restricted the function of the second interface not to need ivar, but that's kind of hackish. Only if the IVARs for interface A and interface B have overlapping values. If the Ivar keys don't overlap, then there's no problems at all. Certainly less hackish than not using them at all. Since dev0 knows the layout of the ivar that it set on its child, this presents no problems at all. It would return the values from A from the right part of the ivar, and those from B in the right part. Apart from the coordination of Ivar numbers, as I outlined in my last post, there's no issue here. I think we should not be talking about the same API here. I have no idea what you mean by the key to value translation, nor Ivar numbers. What I refer to is that device_set_ivars() / device_get_ivars() acts on a single instance variables from `struct device': `ivars'. In that case, I do not really see how to set that specific field to two distinct values for each interfaces. We are talking about the ivar interface. You are just misunderstanding how it is used. yes I indeed did... silly, silly me :-) Actually, no. I wasn't that silly, neither was I misunderstanding anything beside how *you* wanted it to be used, which is, I sorry to say, unacceptable. The last thing I want is to pollute an interface with a single-purpose, hand-crafted, bus. I was to just throw away all that ivar stuff and go into hinted child configuration for now, waiting for FDT... but of course, I figured out after a few hours that hinted child attachment requires `bus_hinted_child' to be set in the parent, as does bus_enumerate_hinted_children() / bus_generic_attach() to explicitly pollute my code. All this stuff should be done implicitly to support N:1 interfaces/client relationship. N *independent* interfaces being provided by a single driver; of course, I'm not even going back to require those interface being provided by multiple drivers, it is already a dead end. I am not even sure any driver in the tree provides more than one interface... For whatever reason, I am more and more thinking that this all new-bus[0] stuff is *way* overkill, static, bloated at will, and missing critical features; a huge PITA to use, for the intended purpose. /me pissed. - Arnaud [0]: damn, why is it even called newbus, this stuff is 14 years old. It really belongs to a museum, not production code... ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 9, 2012 at 11:27 AM, John Baldwin j...@freebsd.org wrote: Also, I think we should do this in general. We already have one example (e.g. ACPI IVARs start at 100 so that things like the ACPI PCI bus driver can provide both ACPI and PCI IVARs to child devices). I think we should assign each bus it's own IVAR range. It would make it easier to determine if a specific IVAR is event supported. Certainly I think ISA and PCI at a minimum should be changed to start at, say, 200 and 300. What's the point doing that ? Let's just resign to the conclusion that FreeBSD devices' subsystem provides no dynamic way for a client device to deals with multiple bus driver. Instead all possible combination have to be harcoded and hand-crafted, when at all possible, to look like they're coming from a single bus... But again, newbus is 14 years old... - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
newbus' ivar's limitation..
Hi folks, Ok, yet another Newbus' limitation. Assuming a device exports more than one interface, and one of its child has need to use more than one interface, each interfaces cannot register, concurrently, its own ivar. While I try to always have a single child per interface/resource, I need to keep some compatibility with the old way of doing thing (POLA wrt. drivers I cannot/will not convert and userland). So, it would have been nice if ivar had been per-interface, not global and unique to one device. Unless I am mistaken, ivar are the only way for a parent can transmit information to a child. I can not simply implement a new METHOD to get that ivar as the device implements multiple time the same function (actually, up to 4 time for one, 3 for the other, with possible crossovers...), each one physically distinct. Each child is being tied to a pair. Thus, I need to pass each child discriminator(s) for each interfaces right after having been *created*, which cannot be done later on. Of course, it is out-of-question to have crossover in the interfaces definitions. The best way I could achieve this currently is to pass the child's device to its parent, and do a lookup based on that pointer to get information I need, but erk my 1c, - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi,
On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh i...@bsdimp.com wrote:
On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote:
Ok, yet another Newbus' limitation. Assuming a device exports more
than one interface, and one of its child has need to use more than one
interface, each interfaces cannot register, concurrently, its own
ivar. While I try to always have a single child per
interface/resource, I need to keep some compatibility with the old way
of doing thing (POLA wrt. drivers I cannot/will not convert and
userland). So, it would have been nice if ivar had been per-interface,
not global and unique to one device.
There's one pointer for the ivars. The bus code gets to determine what the
ivar looks like, because the interface is totally private to the bus. So
long as it returns the right thing for any key that's presented, it doesn't
matter quite how things are done.
So I'm not sure I understand what you are saying here.
dev0 implements two interfaces: A and B. dev1, child of dev0, needs to
use both interfaces. There is no generic way for dev0 to export
independent ivars for both interface. For now, I restricted the
function of the second interface not to need ivar, but that's kind of
hackish.
- Arnaud
The problem, more basically, is that the ivar keys are not unique.
Currently, there's no bits used in the key to define the values to be
non-overlapping. For example:
enum pci_device_ivars {
PCI_IVAR_SUBVENDOR,
PCI_IVAR_SUBDEVICE,
PCI_IVAR_VENDOR,
};
We could easily reserve the upper 16-bits of this field to be that key. This
value could then be used to differentiate them. But this wouldn't scale too
well. Given that there's only about a dozen or two in the tree, that's right
at the moment, it wouldn't be hard to do something like:
enum ivar_namespace {
IVAR_PCI = 1,
IVAR_PCCARD,
IVAR_USB,
etc
};
#define IVAR_SHIFT 16
and the above could be changed to:
enum pci_device_ivars {
PCI_IVAR_SUBVENDOR = IVAR_PCI IVAR_SHIFT,
PCI_IVAR_SUBDEVICE,
PCI_IVAR_VENDOR,
};
and then we'd have an unambiguous key, and the bus could easily implement
multiple interfaces.
but then again, most of the existing interfaces in the kernel are mutually
exclusive, so you could implement this just for your new interfaces.
Unless I am mistaken, ivar are the only way for a parent can transmit
information to a child. I can not simply implement a new METHOD to get
that ivar as the device implements multiple time the same function
(actually, up to 4 time for one, 3 for the other, with possible
crossovers...), each one physically distinct. Each child is being tied
to a pair. Thus, I need to pass each child discriminator(s) for each
interfaces right after having been *created*, which cannot be done
later on. Of course, it is out-of-question to have crossover in the
interfaces definitions.
ivars are but one way to communicate this. However, they are the generic way
to convert a key to a value and store a key on a value. I don't really
understand what you are trying to say here, perhaps an example would help
illustrate what you are trying to do, since I don't quite understand the
problem here.
The best way I could achieve this currently is to pass the child's
device to its parent, and do a lookup based on that pointer to get
information I need, but erk
That doesn't make any sense. The child's parent already sets that child's
ivar when the child is created. The child's parent already gets a pointer to
the child when asked to do the key to value translation. Again, perhaps an
example would help here.
Warner
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
On Sun, Jul 8, 2012 at 11:31 PM, Warner Losh i...@bsdimp.com wrote:
On Jul 8, 2012, at 9:26 PM, Arnaud Lacombe wrote:
Hi,
On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh i...@bsdimp.com wrote:
On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote:
Ok, yet another Newbus' limitation. Assuming a device exports more
than one interface, and one of its child has need to use more than one
interface, each interfaces cannot register, concurrently, its own
ivar. While I try to always have a single child per
interface/resource, I need to keep some compatibility with the old way
of doing thing (POLA wrt. drivers I cannot/will not convert and
userland). So, it would have been nice if ivar had been per-interface,
not global and unique to one device.
There's one pointer for the ivars. The bus code gets to determine what the
ivar looks like, because the interface is totally private to the bus. So
long as it returns the right thing for any key that's presented, it doesn't
matter quite how things are done.
So I'm not sure I understand what you are saying here.
dev0 implements two interfaces: A and B. dev1, child of dev0, needs to
use both interfaces. There is no generic way for dev0 to export
independent ivars for both interface. For now, I restricted the
function of the second interface not to need ivar, but that's kind of
hackish.
Only if the IVARs for interface A and interface B have overlapping values.
If the Ivar keys don't overlap, then there's no problems at all. Certainly
less hackish than not using them at all. Since dev0 knows the layout of the
ivar that it set on its child, this presents no problems at all. It would
return the values from A from the right part of the ivar, and those from B in
the right part. Apart from the coordination of Ivar numbers, as I outlined
in my last post, there's no issue here.
I think we should not be talking about the same API here. I have no
idea what you mean by the key to value translation, nor Ivar
numbers. What I refer to is that device_set_ivars() /
device_get_ivars() acts on a single instance variables from `struct
device': `ivars'. In that case, I do not really see how to set that
specific field to two distinct values for each interfaces.
- Arnaud
Warner
- Arnaud
The problem, more basically, is that the ivar keys are not unique.
Currently, there's no bits used in the key to define the values to be
non-overlapping. For example:
enum pci_device_ivars {
PCI_IVAR_SUBVENDOR,
PCI_IVAR_SUBDEVICE,
PCI_IVAR_VENDOR,
};
We could easily reserve the upper 16-bits of this field to be that key.
This value could then be used to differentiate them. But this wouldn't
scale too well. Given that there's only about a dozen or two in the tree,
that's right at the moment, it wouldn't be hard to do something like:
enum ivar_namespace {
IVAR_PCI = 1,
IVAR_PCCARD,
IVAR_USB,
etc
};
#define IVAR_SHIFT 16
and the above could be changed to:
enum pci_device_ivars {
PCI_IVAR_SUBVENDOR = IVAR_PCI IVAR_SHIFT,
PCI_IVAR_SUBDEVICE,
PCI_IVAR_VENDOR,
};
and then we'd have an unambiguous key, and the bus could easily implement
multiple interfaces.
but then again, most of the existing interfaces in the kernel are mutually
exclusive, so you could implement this just for your new interfaces.
Unless I am mistaken, ivar are the only way for a parent can transmit
information to a child. I can not simply implement a new METHOD to get
that ivar as the device implements multiple time the same function
(actually, up to 4 time for one, 3 for the other, with possible
crossovers...), each one physically distinct. Each child is being tied
to a pair. Thus, I need to pass each child discriminator(s) for each
interfaces right after having been *created*, which cannot be done
later on. Of course, it is out-of-question to have crossover in the
interfaces definitions.
ivars are but one way to communicate this. However, they are the generic
way to convert a key to a value and store a key on a value. I don't really
understand what you are trying to say here, perhaps an example would help
illustrate what you are trying to do, since I don't quite understand the
problem here.
The best way I could achieve this currently is to pass the child's
device to its parent, and do a lookup based on that pointer to get
information I need, but erk
That doesn't make any sense. The child's parent already sets that child's
ivar when the child is created. The child's parent already gets a pointer
to the child when asked to do the key to value translation. Again, perhaps
an example would help here.
Warner
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi,
On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh i...@bsdimp.com wrote:
On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote:
Ok, yet another Newbus' limitation. Assuming a device exports more
than one interface, and one of its child has need to use more than one
interface, each interfaces cannot register, concurrently, its own
ivar. While I try to always have a single child per
interface/resource, I need to keep some compatibility with the old way
of doing thing (POLA wrt. drivers I cannot/will not convert and
userland). So, it would have been nice if ivar had been per-interface,
not global and unique to one device.
There's one pointer for the ivars. The bus code gets to determine what the
ivar looks like, because the interface is totally private to the bus. So
long as it returns the right thing for any key that's presented, it doesn't
matter quite how things are done.
So I'm not sure I understand what you are saying here.
The problem, more basically, is that the ivar keys are not unique.
Currently, there's no bits used in the key to define the values to be
non-overlapping. For example:
enum pci_device_ivars {
PCI_IVAR_SUBVENDOR,
PCI_IVAR_SUBDEVICE,
PCI_IVAR_VENDOR,
};
We could easily reserve the upper 16-bits of this field to be that key. This
value could then be used to differentiate them. But this wouldn't scale too
well. Given that there's only about a dozen or two in the tree, that's right
at the moment, it wouldn't be hard to do something like:
enum ivar_namespace {
IVAR_PCI = 1,
IVAR_PCCARD,
IVAR_USB,
etc
};
#define IVAR_SHIFT 16
and the above could be changed to:
enum pci_device_ivars {
PCI_IVAR_SUBVENDOR = IVAR_PCI IVAR_SHIFT,
PCI_IVAR_SUBDEVICE,
PCI_IVAR_VENDOR,
};
and then we'd have an unambiguous key, and the bus could easily implement
multiple interfaces.
but then again, most of the existing interfaces in the kernel are mutually
exclusive, so you could implement this just for your new interfaces.
ok, I think I got it now. You and I are exactly saying the same thing,
just in different terms; there is no way to currently specify multiple
independent (/overlapping) ivars in a child...
- Arnaud
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: newbus' ivar's limitation..
Hi, On Mon, Jul 9, 2012 at 12:37 AM, Warner Losh i...@bsdimp.com wrote: On Jul 8, 2012, at 9:46 PM, Arnaud Lacombe wrote: On Sun, Jul 8, 2012 at 11:31 PM, Warner Losh i...@bsdimp.com wrote: On Jul 8, 2012, at 9:26 PM, Arnaud Lacombe wrote: Hi, On Sun, Jul 8, 2012 at 10:07 PM, Warner Losh i...@bsdimp.com wrote: On Jul 8, 2012, at 7:22 PM, Arnaud Lacombe wrote: Ok, yet another Newbus' limitation. Assuming a device exports more than one interface, and one of its child has need to use more than one interface, each interfaces cannot register, concurrently, its own ivar. While I try to always have a single child per interface/resource, I need to keep some compatibility with the old way of doing thing (POLA wrt. drivers I cannot/will not convert and userland). So, it would have been nice if ivar had been per-interface, not global and unique to one device. There's one pointer for the ivars. The bus code gets to determine what the ivar looks like, because the interface is totally private to the bus. So long as it returns the right thing for any key that's presented, it doesn't matter quite how things are done. So I'm not sure I understand what you are saying here. dev0 implements two interfaces: A and B. dev1, child of dev0, needs to use both interfaces. There is no generic way for dev0 to export independent ivars for both interface. For now, I restricted the function of the second interface not to need ivar, but that's kind of hackish. Only if the IVARs for interface A and interface B have overlapping values. If the Ivar keys don't overlap, then there's no problems at all. Certainly less hackish than not using them at all. Since dev0 knows the layout of the ivar that it set on its child, this presents no problems at all. It would return the values from A from the right part of the ivar, and those from B in the right part. Apart from the coordination of Ivar numbers, as I outlined in my last post, there's no issue here. I think we should not be talking about the same API here. I have no idea what you mean by the key to value translation, nor Ivar numbers. What I refer to is that device_set_ivars() / device_get_ivars() acts on a single instance variables from `struct device': `ivars'. In that case, I do not really see how to set that specific field to two distinct values for each interfaces. We are talking about the ivar interface. You are just misunderstanding how it is used. yes I indeed did... silly, silly me :-) - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Interfacing devices with multiple parents within newbus
Hi,
On Sat, Jul 7, 2012 at 2:47 PM, Ian Lepore
free...@damnhippie.dyndns.org wrote:
On Fri, 2012-07-06 at 16:45 -0400, Arnaud Lacombe wrote:
Hi,
On Fri, Jul 6, 2012 at 3:09 PM, Ian Lepore
free...@damnhippie.dyndns.org wrote:
On Fri, 2012-07-06 at 14:46 -0400, Arnaud Lacombe wrote:
Hi,
On Fri, Jul 6, 2012 at 11:33 AM, Arnaud Lacombe lacom...@gmail.com
wrote:
That's neither correct nor robust in a couple of way:
1) you have no guarantee a device unit will always give you the same
resource.
this raises the following question: how can a device, today, figure
out which parent in a given devclass would give it access to resources
it needs.
Say, you have gpiobus0 provided by a superio and gpiobus1 provided by
the chipset and a LED on the chipset's GPIO. Now, say gpiobus0
attachment is conditional to some BIOS setting. How can you tell
gpioled(4) to attach on the chipset provided GPIO without hardcoding
unit number either way ?
AFAIK, you can not.
Even hints provided layout description is defeated. Each device in a
given devclass need to have a set of unique attribute to allow a child
to distinguish it from other potential parent in the same devclass...
- Arnaud
Talking about a child being unable to choose the correct parent seems to
indicate that this whole problem is turned upside-down somehow; children
don't choose their parents.
actually, I think I was wrong, I thought device were attached to a
devclass, but they are truly attached to a given device. My mistake.
Just blue-sky dreaming here on the fly... what we really have is a
resource-management problem. A device comes along that needs a GPIO
resource, how does it find and use that resource?
Well, we have a resource manager, could that help somehow? Could a
driver that provides access to GPIO somehow register its availability so
that another driver can find and access it? The resource may be a
callable interface, it doesn't really matter, I'm just wondering if the
current rman stuff could be leveraged to help make the connection
between unrelated devices. I think that implies that there would have
to be something near the root of the hiearchy willing to be the
owner/manager of dynamic resources.
AFAIR, rman is mostly there to manage memory vs. i/o mapped resources.
The more I think about it, the more FTD is the answer. The open
question now being how to map a flexible device structure (FTD) to a
less flexible structure (Newbus) :/
- Arnaud
Memory- and IO-mapped regions and IRQs are the only current uses of rman
(that I know of), but it was designed to be fairly agnostic about the
resources it manages. It just works with ranges of values (that it
really doesn't know how to interpret at all), leaving lots of room to
define new types of things it can manage.
The downside is that it's designed to be used hierarchically in the
context of newbus, specifically to help parents manage the resources
that they are able to provide to their children. Trying to use it in a
way that allows devices which are hierarchically unrelated to allocate
resources from each other may amount to a square-peg/round-hole
situation. But the alternative is writing a new facility to allow
registration and allocation of resources using some sort symbolic method
of representing the resources such that the new manager doesn't have to
know much about what it's managing. I think it would be better to find
a way to reuse what we've already got if that's possible.
I think we have two semi-related aspects to this problem...
How do we symbolically represent the resources that drivers can provide
to each other? (FDT may be the answer; I don't know much about it.)
How do devices use that symbolic representation to locate the provider
of the resource, and how is the sharing of those resources managed?
I'd say FDT answer both question, take the DTS for Freescale i.MX53
Smart Mobile Reference Design Board[0],
We first have SoC generic definition in `imx53.dtsi':
soc {
compatible = simple-bus;
aips@5000 { /* AIPS1 */
compatible = fsl,aips-bus, simple-bus;
spba@5000 {
compatible = fsl,spba-bus, simple-bus;
esdhc@50004000 { /* ESDHC1 */
compatible = fsl,imx53-esdhc;
reg = 0x50004000 0x4000;
interrupts = 1;
status = disabled;
};
[...]
gpio2: gpio@53f8c000 { /* GPIO3 */
compatible = fsl,imx53-gpio, fsl,imx31-gpio;
reg = 0x53f8c000 0x4000;
};
gpio3: gpio@53f9 { /* GPIO4 */
compatible = fsl,imx53-gpio, fsl,imx31-gpio;
reg = 0x53f9 0x4000;
};
[...]
}
then board specific definition overriding its parent's in `imx53-smd.dts':
soc {
aips@5000 { /* AIPS1 */
spba@5000 {
esdhc@50004000 { /* ESDHC1 */
cd-gpios = gpio2 13 0; /* GPIO3_13 */
wp-gpios = gpio3 11 0; /* GPIO4_11 */
status = okay
Re: Interfacing devices with multiple parents within newbus
Hi,
On Fri, Jul 6, 2012 at 1:04 AM, Warner Losh i...@bsdimp.com wrote:
On Jul 5, 2012, at 5:14 PM, Arnaud Lacombe wrote:
Hi folks,
The problem has been raised in the last BSDCan during a talk, but no
clear answer has been given. Some (pseudo-)devices might require
resources from multiple other (pseudo-)devices.
For example, a device is sitting on an SMBus, but need to access a
software controlled LED, sitting on a GPIO bus, itself sitting on an
LPC bus... Or a variant where everything is controlled on the same
chip, but different GPIO banks. For that specific example, all the
GPIO pin could be implement on the same GPIObus, however, gpiobus(4)
is limited to 32 pins and the chip provides 5 banks of 8 pins, ie. a
total of 40 pins[0]. In the same idea, a device sitting on GPIOs
controlled by two independant chips, say one being an ICH10R chipset
on a PCI device function, and the other being a
SuperIO on an LPC bus.
This situation make me really dubious that FreeBSD will be able to
support configuration such as:
esdhc@50004000 { /* ESDHC1 */
cd-gpios = gpio2 13 0; /* GPIO3_13 */
wp-gpios = gpio3 11 0; /* GPIO4_11 */
status = okay;
};
or:
ecspi@5001 { /* ECSPI1 */
fsl,spi-num-chipselects = 2;
cs-gpios = gpio1 30 0, /* GPIO2_30 */
gpio2 19 0; /* GPIO3_19 */
status = okay;
[...]
This example is taken from Linux' `arch/arm/boot/dts/imx53-smd.dts'.
Here, SDHC or SPI controller are using different GPIO devices. Note
that these GPIO pins does not seem to be multi-function pins as
another .dts defines ESDHC1 as:
esdhc@50004000 { /* ESDHC1 */
cd-gpios = gpio2 13 0; /* GPIO3_13 */
wp-gpios = gpio2 14 0; /* GPIO3_14 */
status = okay;
};
AFAIK, newbus is unable to model any of the above situation without
being bypassed one way or another.
any hints ?
That's not correct. You don't need a parent-child relationship in newbus to
interact with another node in the tree. You can look up the other node by
name, and then call functions based on finding that other device.
I assume you are talking about devclass_get_device()/device_find_child().
That's neither correct nor robust in a couple of way:
1) you have no guarantee a device unit will always give you the same resource.
2) there is no reference counting on the returned device.
3) there is no track record of the reference being given.
About (1), lower unit devices can fails to attach[0], thus newly
attached bus will now have a negative offset.
About (2) and (3), referenced device (think KLD) might go away and the
child will not be told. In this situation, I want the child to be
detached prior to its parent.
As such, looking up other node by name would fit in what I call
bypassing newbus purpose. I might just as well export a damn
function pointer and make my life easier.
What would be need is an interface where the child need to have a way
to say resources I need are not complete, tell me when a new device
in devclass Z is attached that I can check if it fills my need, ala:
- dev0 is probed
- dev0 tells it is present, but need unavailable resource in devclass
X and devclass Y
- dev0 is marked as pending
- dev1 is attached in devclass X
- dev0 is asked if dev1 would suit it
- dev0 answers negatively
- dev2 is attached in devclass X
- dev0 is asked if dev2 would suit it
- dev0 answers positively
- dev0 continues its probe, but still need unavailable resource in devclass Y
- dev3 is attached in devclass Y
- dev0 is asked if dev3 would suit it
- dev0 answers positively
- dev0 finishes its probe
- dev0 is attached
[...]
- dev3 is to be detached
- dev0 is detached
- dev3 is detached
- Arnaud
[0]: for many reason; hardware failure, configuration change (think
GPIO on multi-function pin)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Interfacing devices with multiple parents within newbus
Hi, On Fri, Jul 6, 2012 at 11:33 AM, Arnaud Lacombe lacom...@gmail.com wrote: That's neither correct nor robust in a couple of way: 1) you have no guarantee a device unit will always give you the same resource. this raises the following question: how can a device, today, figure out which parent in a given devclass would give it access to resources it needs. Say, you have gpiobus0 provided by a superio and gpiobus1 provided by the chipset and a LED on the chipset's GPIO. Now, say gpiobus0 attachment is conditional to some BIOS setting. How can you tell gpioled(4) to attach on the chipset provided GPIO without hardcoding unit number either way ? AFAIK, you can not. Even hints provided layout description is defeated. Each device in a given devclass need to have a set of unique attribute to allow a child to distinguish it from other potential parent in the same devclass... - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Interfacing devices with multiple parents within newbus
Hi, On Fri, Jul 6, 2012 at 3:09 PM, Ian Lepore free...@damnhippie.dyndns.org wrote: On Fri, 2012-07-06 at 14:46 -0400, Arnaud Lacombe wrote: Hi, On Fri, Jul 6, 2012 at 11:33 AM, Arnaud Lacombe lacom...@gmail.com wrote: That's neither correct nor robust in a couple of way: 1) you have no guarantee a device unit will always give you the same resource. this raises the following question: how can a device, today, figure out which parent in a given devclass would give it access to resources it needs. Say, you have gpiobus0 provided by a superio and gpiobus1 provided by the chipset and a LED on the chipset's GPIO. Now, say gpiobus0 attachment is conditional to some BIOS setting. How can you tell gpioled(4) to attach on the chipset provided GPIO without hardcoding unit number either way ? AFAIK, you can not. Even hints provided layout description is defeated. Each device in a given devclass need to have a set of unique attribute to allow a child to distinguish it from other potential parent in the same devclass... - Arnaud Talking about a child being unable to choose the correct parent seems to indicate that this whole problem is turned upside-down somehow; children don't choose their parents. actually, I think I was wrong, I thought device were attached to a devclass, but they are truly attached to a given device. My mistake. Just blue-sky dreaming here on the fly... what we really have is a resource-management problem. A device comes along that needs a GPIO resource, how does it find and use that resource? Well, we have a resource manager, could that help somehow? Could a driver that provides access to GPIO somehow register its availability so that another driver can find and access it? The resource may be a callable interface, it doesn't really matter, I'm just wondering if the current rman stuff could be leveraged to help make the connection between unrelated devices. I think that implies that there would have to be something near the root of the hiearchy willing to be the owner/manager of dynamic resources. AFAIR, rman is mostly there to manage memory vs. i/o mapped resources. The more I think about it, the more FTD is the answer. The open question now being how to map a flexible device structure (FTD) to a less flexible structure (Newbus) :/ - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Interfacing devices with multiple parents within newbus
Hi folks,
The problem has been raised in the last BSDCan during a talk, but no
clear answer has been given. Some (pseudo-)devices might require
resources from multiple other (pseudo-)devices.
For example, a device is sitting on an SMBus, but need to access a
software controlled LED, sitting on a GPIO bus, itself sitting on an
LPC bus... Or a variant where everything is controlled on the same
chip, but different GPIO banks. For that specific example, all the
GPIO pin could be implement on the same GPIObus, however, gpiobus(4)
is limited to 32 pins and the chip provides 5 banks of 8 pins, ie. a
total of 40 pins[0]. In the same idea, a device sitting on GPIOs
controlled by two independant chips, say one being an ICH10R chipset
on a PCI device function, and the other being a
SuperIO on an LPC bus.
This situation make me really dubious that FreeBSD will be able to
support configuration such as:
esdhc@50004000 { /* ESDHC1 */
cd-gpios = gpio2 13 0; /* GPIO3_13 */
wp-gpios = gpio3 11 0; /* GPIO4_11 */
status = okay;
};
or:
ecspi@5001 { /* ECSPI1 */
fsl,spi-num-chipselects = 2;
cs-gpios = gpio1 30 0, /* GPIO2_30 */
gpio2 19 0; /* GPIO3_19 */
status = okay;
[...]
This example is taken from Linux' `arch/arm/boot/dts/imx53-smd.dts'.
Here, SDHC or SPI controller are using different GPIO devices. Note
that these GPIO pins does not seem to be multi-function pins as
another .dts defines ESDHC1 as:
esdhc@50004000 { /* ESDHC1 */
cd-gpios = gpio2 13 0; /* GPIO3_13 */
wp-gpios = gpio2 14 0; /* GPIO3_14 */
status = okay;
};
AFAIK, newbus is unable to model any of the above situation without
being bypassed one way or another.
any hints ?
Thanks,
- Arnaud
[0]: as a matter of comparison, old PXA27x have up to 121 GPIOs
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
sysctl filesystem ?
Hi folks, I find myself in a situation where I need to directly explore the sysctl(8) tree from my program. The tricky part is this: from `src/sbin/sysctl.c': /* * These functions uses a presently undocumented interface to the kernel * to walk the tree and get the type so it can print the value. * This interface is under work and consideration, and should probably * be killed with a big axe by the first person who can find the time. * (be aware though, that the proper interface isn't as obvious as it * may seem, there are various conflicting requirements. */ AFAIT, the whole interface used by sysctl(8) to explore the sysctl tree (ie. list, name, get description) is undocumented. This comment has been there for about, well... 17 years. No matter to say that it is highly unlikely anyone is ever gonna design that perfect interface. Right now, I am left with no choice but to figure out how that stuff work, which I foresee will be a real PITA. No choice ? Well, not so much. About 12 years ago a filesystem interface was written for sysctl, namely scfs(4). It was authored by Kelly Yancey and (?) Boris Popov. Unfortunately, time passed and no code is any longer publicly available. URLs are either no longer valid or password-protected. This interface would just be perfect for my use-case. No need to spend time decoding a prehistoric interface, no need to craft custom accessors. I would just have to use standard POSIX file interface... if the code was available. I was hoping some of the people on this list would either, have the scfs(4) code on their drives/tapes, or have a way to contact the original author(s) and thus have a chance to get that code ? Thanks in advance, - Arnaud ps: I know the code will certainly have to be fixed, but that is still the best option I've got so far. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl filesystem ?
Hi, On Mon, Jun 25, 2012 at 8:30 PM, Adam Vande More amvandem...@gmail.com wrote: On Mon, Jun 25, 2012 at 7:03 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi folks, I find myself in a situation where I need to directly explore the sysctl(8) tree from my program. The tricky part is this: There is this: http://svnweb.freebsd.org/base/releng/4.7/sys/miscfs/kernfs/ yes, `kernfs' is mentioned in some of the thread about a sysctl filesystem as a potential code base, and has been used in such purpose. However, if I can avoid to re-design that wheel too, by getting access to scfs(4) code, I will. - Arnaud -- Adam Vande More ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl filesystem ?
On Mon, Jun 25, 2012 at 8:13 PM, Garrett Cooper yaneg...@gmail.com wrote:
On Mon, Jun 25, 2012 at 5:03 PM, Arnaud Lacombe lacom...@gmail.com wrote:
Hi folks,
I find myself in a situation where I need to directly explore the
sysctl(8) tree from my program. The tricky part is this:
from `src/sbin/sysctl.c':
/*
* These functions uses a presently undocumented interface to the kernel
* to walk the tree and get the type so it can print the value.
* This interface is under work and consideration, and should probably
* be killed with a big axe by the first person who can find the time.
* (be aware though, that the proper interface isn't as obvious as it
* may seem, there are various conflicting requirements.
*/
AFAIT, the whole interface used by sysctl(8) to explore the sysctl
tree (ie. list, name, get description) is undocumented. This comment
has been there for about, well... 17 years. No matter to say that it
is highly unlikely anyone is ever gonna design that perfect interface.
Right now, I am left with no choice but to figure out how that stuff
work, which I foresee will be a real PITA. No choice ? Well, not so
much. About 12 years ago a filesystem interface was written for
sysctl, namely scfs(4). It was authored by Kelly Yancey and (?) Boris
Popov. Unfortunately, time passed and no code is any longer publicly
available. URLs are either no longer valid or password-protected.
This interface would just be perfect for my use-case. No need to spend
time decoding a prehistoric interface, no need to craft custom
accessors. I would just have to use standard POSIX file interface...
if the code was available.
I was hoping some of the people on this list would either, have the
scfs(4) code on their drives/tapes, or have a way to contact the
original author(s) and thus have a chance to get that code ?
Thanks in advance,
- Arnaud
ps: I know the code will certainly have to be fixed, but that is still
the best option I've got so far.
Alternatively, the interface could just be documented (from
sys/kern/kern_sysctl.c):
/*
* Staff-functions
*
* These functions implement a presently undocumented interface
* used by the sysctl program to walk the tree, and get the type
* so it can print the value.
* This interface is under work and consideration, and should probably
* be killed with a big axe by the first person who can find the time.
* (be aware though, that the proper interface isn't as obvious as it
* may seem, there are various conflicting requirements.
*
* {0,0} printf the entire MIB-tree.
* {0,1,...} return the name of the ... OID.
* {0,2,...} return the next OID.
* {0,3} return the OID of the name in new
* {0,4,...} return the kind format info for the ... OID.
* {0,5,...} return the description the ... OID.
*/
I know 2 closed-source versions that have wholesale stolen/borrowed
the code from sysctl(3), and I adapted the sysctl(8) code for my own
uses for the cython derivative I made here [1].
I guess I will have no choice but to write a third...
- Arnaud
Thanks,
-Garrett
1. http://sourceforge.net/projects/sysctl/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl filesystem ?
Hi, On Mon, Jun 25, 2012 at 10:51 PM, Boris Popov b...@freebsd.org wrote: On 26.06.2012 6:56, Arnaud Lacombe wrote: purpose. However, if I can avoid to re-design that wheel too, by getting access to scfs(4) code, I will. It is interesting, that the old drive with this code are still alive. Most likely, FS related part will need serious attention because of numerous changes in the VFS subsystem. Here is the link: http://www.vertex.kz/scfs.tgz Outstanding ! I was pointed another implementation by Jakub Dawidek made in 2002, which seems more advanced. In any case, lots of KPI changed... I guess I found a new toy for this week :-) Thanks a lot, - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: fast bcopy...
Hi, On Wed, May 2, 2012 at 5:52 PM, Steven Atreju snatr...@googlemail.com wrote: Luigi Rizzo wrote: 2. apparently, bcopy is not the fastest way to copy memory. http://now.cs.berkeley.edu/Td/bcopy.html Pentium 166, Triton Chipset, EDO memory... ahem. - Arnaud Best Regards. Steven. ___ freebsd-...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to freebsd-net-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Complete hang on 9.0-RELEASE
Hi, On Sat, Apr 21, 2012 at 4:19 AM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Wed, Apr 18, 2012 at 2:22 AM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 5:50 PM, Arnaud Lacombe lacom...@gmail.com wrote: [...] I reproduced the previous problem on 10-CURRENT from r233917, on the following platform (here running 8.2-RELEASE): FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 r...@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Timecounter i8254 frequency 1193182 Hz quality 0 CPU: Intel(R) Atom(TM) CPU D525 @ 1.80GHz (1800.01-MHz K8-class CPU) Origin = GenuineIntel Id = 0x106ca Family = 6 Model = 1c Stepping = 10 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x40e31dSSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE AMD Features=0x20100800SYSCALL,NX,LM AMD Features2=0x1LAHF TSC: P-state invariant real memory = 2136539136 (2037 MB) avail memory = 2043772928 (1949 MB) ACPI APIC Table: 010312 APIC0947 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 Complete system freeze while running about 2400 threads. I had to power cycle the system to get it back alive. I discussed a way to debug this with attilio@ on freebsd-stable@, but still did not had time to implement it. 10-CURRENT from r233917 hanged again today while running 3600 threads. I enabled WITNESS and INVARIANTS on that specific kernel, secretly hoping that they would trigger some meaningful information, but they did not. I would guess my last attempt is to enable SW_WATCHDOG, and gather some state information out of DDB when the watchdog trigger, if it does... Btw, this issue seems to be specifically happening on Atom/ICH8M platform running amd64 kernel, as I've never seen it on other platforms, and yet ran extensive tests. I am not entirely sure it happens on i386. I would need to check. For the record, 9.0-RELEASE i386 has been running the test for about 2 days on the D510 platform without any hang so far. I'll keep it running all week-end to give me a better idea. ... or I have been too eager to expect an amd64 only issue. Thanks to some nasty virus which stuck me in my bed for two days, I finally got FreeBSD 9.0-RELEASE i386 stuck while running a single, 4000 threads, process. I guess it's time to play with SW_WATCHDOG and DDB. As a side note, the D510 platform seem to be much harder to hang than the D525... - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Disabling an arbitrary device
Hi, On Fri, Apr 20, 2012 at 8:50 PM, Attilio Rao atti...@freebsd.org wrote: Il 20 aprile 2012 19:18, Arnaud Lacombe lacom...@gmail.com ha scritto: Hi, On Fri, Apr 20, 2012 at 2:16 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, I will be bringing up an old thread there, but it would seem the situation did not evolve in the past 9 years. I have a machine running 7.1 whose UHCI controller is generating some interrupt storm: # vmstat -i interrupt total rate irq4: sio0 1328 2 irq19: uhci1+ 63514509 96380 [...] generating useless load on one CPU: # top -SH last pid: 5223; load averages: 0.00, 0.00, 0.00 up 0+00:17:21 13:10:35 117 processes: 14 running, 79 sleeping, 24 waiting CPU: 0.2% user, 0.0% nice, 0.2% system, 6.6% interrupt, 93.0% idle Mem: 33M Active, 9348K Inact, 67M Wired, 400K Cache, 29M Buf, 2892M Free [...] 57 root -64 - 0K 8K CPU0 0 11:59 86.57% irq19: uhci1+ I thought I could use an hint to forbid uhci(4) attachment, ala: hint.uhci.0.disabled=1 hint.uhci.1.disabled=1 in /boot/loader.conf. However, it would seem that what should be usable with any arbitrary devices, ie. be an integral part of device(9), has to be hardcoded in every driver, sad. as for the usual if you're not happy, please provide a patch: https://github.com/lacombar/freebsd/commit/30786d09b0cb441890cdc749ee5243238e81d2d8 I think a generalized kludge should really belong to device_probe_child() rather than device_add_child_ordered(). suit me. When you have a tested patch against -CURRENT, can you please send to freebsd-newbus@? will give it a try, eventually. BTW, IIRC 7.x doesn't have the new USB stack, which means that you can have caught easilly a driver bug there, did you consider switching at least to 8.x kernel? That is unfortunately not a decision for me to make, though a switch to 9.x will be more likely. That said, it seems that it is not a USB specific issue, but an IRQ19's issue. I did not had a chance to investigate that further, but with uhci(4) disabled, atapci(4) is now IRQ-storming. This is a known issue relatively widely encountered, with various workarounds possible. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Complete hang on 9.0-RELEASE
Hi, On Wed, Apr 18, 2012 at 2:22 AM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 5:50 PM, Arnaud Lacombe lacom...@gmail.com wrote: [...] I reproduced the previous problem on 10-CURRENT from r233917, on the following platform (here running 8.2-RELEASE): FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 r...@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Timecounter i8254 frequency 1193182 Hz quality 0 CPU: Intel(R) Atom(TM) CPU D525 @ 1.80GHz (1800.01-MHz K8-class CPU) Origin = GenuineIntel Id = 0x106ca Family = 6 Model = 1c Stepping = 10 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x40e31dSSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE AMD Features=0x20100800SYSCALL,NX,LM AMD Features2=0x1LAHF TSC: P-state invariant real memory = 2136539136 (2037 MB) avail memory = 2043772928 (1949 MB) ACPI APIC Table: 010312 APIC0947 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 Complete system freeze while running about 2400 threads. I had to power cycle the system to get it back alive. I discussed a way to debug this with attilio@ on freebsd-stable@, but still did not had time to implement it. 10-CURRENT from r233917 hanged again today while running 3600 threads. I enabled WITNESS and INVARIANTS on that specific kernel, secretly hoping that they would trigger some meaningful information, but they did not. I would guess my last attempt is to enable SW_WATCHDOG, and gather some state information out of DDB when the watchdog trigger, if it does... Btw, this issue seems to be specifically happening on Atom/ICH8M platform running amd64 kernel, as I've never seen it on other platforms, and yet ran extensive tests. I am not entirely sure it happens on i386. I would need to check. For the record, 9.0-RELEASE i386 has been running the test for about 2 days on the D510 platform without any hang so far. I'll keep it running all week-end to give me a better idea. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Disabling an arbitrary device
Hi, I will be bringing up an old thread there, but it would seem the situation did not evolve in the past 9 years. I have a machine running 7.1 whose UHCI controller is generating some interrupt storm: # vmstat -i interrupt total rate irq4: sio0 1328 2 irq19: uhci1+ 63514509 96380 [...] generating useless load on one CPU: # top -SH last pid: 5223; load averages: 0.00, 0.00, 0.00up 0+00:17:21 13:10:35 117 processes: 14 running, 79 sleeping, 24 waiting CPU: 0.2% user, 0.0% nice, 0.2% system, 6.6% interrupt, 93.0% idle Mem: 33M Active, 9348K Inact, 67M Wired, 400K Cache, 29M Buf, 2892M Free [...] 57 root -64- 0K 8K CPU0 0 11:59 86.57% irq19: uhci1+ I thought I could use an hint to forbid uhci(4) attachment, ala: hint.uhci.0.disabled=1 hint.uhci.1.disabled=1 in /boot/loader.conf. However, it would seem that what should be usable with any arbitrary devices, ie. be an integral part of device(9), has to be hardcoded in every driver, sad. - Arnaud ps: the original thread I found is from September 2004: http://lists.freebsd.org/pipermail/freebsd-questions/2004-September/058717.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Disabling an arbitrary device
Hi, On Fri, Apr 20, 2012 at 2:16 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, I will be bringing up an old thread there, but it would seem the situation did not evolve in the past 9 years. I have a machine running 7.1 whose UHCI controller is generating some interrupt storm: # vmstat -i interrupt total rate irq4: sio0 1328 2 irq19: uhci1+ 63514509 96380 [...] generating useless load on one CPU: # top -SH last pid: 5223; load averages: 0.00, 0.00, 0.00 up 0+00:17:21 13:10:35 117 processes: 14 running, 79 sleeping, 24 waiting CPU: 0.2% user, 0.0% nice, 0.2% system, 6.6% interrupt, 93.0% idle Mem: 33M Active, 9348K Inact, 67M Wired, 400K Cache, 29M Buf, 2892M Free [...] 57 root -64 - 0K 8K CPU0 0 11:59 86.57% irq19: uhci1+ I thought I could use an hint to forbid uhci(4) attachment, ala: hint.uhci.0.disabled=1 hint.uhci.1.disabled=1 in /boot/loader.conf. However, it would seem that what should be usable with any arbitrary devices, ie. be an integral part of device(9), has to be hardcoded in every driver, sad. as for the usual if you're not happy, please provide a patch: https://github.com/lacombar/freebsd/commit/30786d09b0cb441890cdc749ee5243238e81d2d8 regards, - Arnaud - Arnaud ps: the original thread I found is from September 2004: http://lists.freebsd.org/pipermail/freebsd-questions/2004-September/058717.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Complete hang on 9.0-RELEASE
Hi, On Mon, Apr 16, 2012 at 5:50 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, [for the record...] On Tue, Feb 14, 2012 at 11:41 AM, Arnaud Lacombe lacom...@gmail.com wrote: Hi folks, For the records, I was running some tests yesterday on top of a 9.0-RELEASE, amd64, kernel when the box hanged. At the time of the hang, the box was running a process with about 2800 threads with heavy IPC between 1400 writers and 1400 readers. The box was in single user mode (/bin/sh coming from FreeBSD 7.4-STABLE). Here is the beginning of the dmesg: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Atom(TM) CPU D510 @ 1.66GHz (1666.70-MHz K8-class CPU) Origin = GenuineIntel Id = 0x106ca Family = 6 Model = 1c Stepping = 10 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x40e31dSSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF TSC: P-state invariant, performance statistics real memory = 2137587712 (2038 MB) avail memory = 2037841920 (1943 MB) Event timer LAPIC quality 400 ACPI APIC Table: 070611 APIC1125 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 I will restart the test and see if this happens again. I reproduced the previous problem on 10-CURRENT from r233917, on the following platform (here running 8.2-RELEASE): FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 r...@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Timecounter i8254 frequency 1193182 Hz quality 0 CPU: Intel(R) Atom(TM) CPU D525 @ 1.80GHz (1800.01-MHz K8-class CPU) Origin = GenuineIntel Id = 0x106ca Family = 6 Model = 1c Stepping = 10 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x40e31dSSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE AMD Features=0x20100800SYSCALL,NX,LM AMD Features2=0x1LAHF TSC: P-state invariant real memory = 2136539136 (2037 MB) avail memory = 2043772928 (1949 MB) ACPI APIC Table: 010312 APIC0947 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 Complete system freeze while running about 2400 threads. I had to power cycle the system to get it back alive. I discussed a way to debug this with attilio@ on freebsd-stable@, but still did not had time to implement it. 10-CURRENT from r233917 hanged again today while running 3600 threads. I enabled WITNESS and INVARIANTS on that specific kernel, secretly hoping that they would trigger some meaningful information, but they did not. I would guess my last attempt is to enable SW_WATCHDOG, and gather some state information out of DDB when the watchdog trigger, if it does... Btw, this issue seems to be specifically happening on Atom/ICH8M platform running amd64 kernel, as I've never seen it on other platforms, and yet ran extensive tests. I am not entirely sure it happens on i386. I would need to check. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Complete hang on 9.0-RELEASE
Hi, [for the record...] On Tue, Feb 14, 2012 at 11:41 AM, Arnaud Lacombe lacom...@gmail.com wrote: Hi folks, For the records, I was running some tests yesterday on top of a 9.0-RELEASE, amd64, kernel when the box hanged. At the time of the hang, the box was running a process with about 2800 threads with heavy IPC between 1400 writers and 1400 readers. The box was in single user mode (/bin/sh coming from FreeBSD 7.4-STABLE). Here is the beginning of the dmesg: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 r...@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 CPU: Intel(R) Atom(TM) CPU D510 @ 1.66GHz (1666.70-MHz K8-class CPU) Origin = GenuineIntel Id = 0x106ca Family = 6 Model = 1c Stepping = 10 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x40e31dSSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE AMD Features=0x2800SYSCALL,LM AMD Features2=0x1LAHF TSC: P-state invariant, performance statistics real memory = 2137587712 (2038 MB) avail memory = 2037841920 (1943 MB) Event timer LAPIC quality 400 ACPI APIC Table: 070611 APIC1125 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 I will restart the test and see if this happens again. I reproduced the previous problem on 10-CURRENT from r233917, on the following platform (here running 8.2-RELEASE): FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.2-RELEASE #0: Thu Feb 17 02:41:51 UTC 2011 r...@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Timecounter i8254 frequency 1193182 Hz quality 0 CPU: Intel(R) Atom(TM) CPU D525 @ 1.80GHz (1800.01-MHz K8-class CPU) Origin = GenuineIntel Id = 0x106ca Family = 6 Model = 1c Stepping = 10 Features=0xbfebfbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE Features2=0x40e31dSSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE AMD Features=0x20100800SYSCALL,NX,LM AMD Features2=0x1LAHF TSC: P-state invariant real memory = 2136539136 (2037 MB) avail memory = 2043772928 (1949 MB) ACPI APIC Table: 010312 APIC0947 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 Complete system freeze while running about 2400 threads. I had to power cycle the system to get it back alive. I discussed a way to debug this with attilio@ on freebsd-stable@, but still did not had time to implement it. regards, - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
FreeBSD on recent Xeon E5
Hi, I just booted FreeBSD 9.0-RELEASE on a Xeon E5-1650 based platform. It would seems that the CPU handles by itself a lots of PCI functions which do not seem to be supported by FreeBSD. Here is the output of `pciconf -l' restricted to unhandled devices: none0@pci0:0:4:0: class=0x088000 card=0x062b15d9 chip=0x3c208086 none1@pci0:0:4:1: class=0x088000 card=0x062b15d9 chip=0x3c218086 none2@pci0:0:4:2: class=0x088000 card=0x062b15d9 chip=0x3c228086 none3@pci0:0:4:3: class=0x088000 card=0x062b15d9 chip=0x3c238086 none4@pci0:0:4:4: class=0x088000 card=0x062b15d9 chip=0x3c248086 none5@pci0:0:4:5: class=0x088000 card=0x062b15d9 chip=0x3c258086 none6@pci0:0:4:6: class=0x088000 card=0x062b15d9 chip=0x3c268086 none7@pci0:0:4:7: class=0x088000 card=0x062b15d9 chip=0x3c278086 none8@pci0:0:5:0: class=0x088000 card=0x062b15d9 chip=0x3c288086 none9@pci0:0:5:2: class=0x088000 card=0x062b15d9 chip=0x3c2a8086 none10@pci0:0:22:0: class=0x078000 card=0x062b15d9 chip=0x1d3a8086 none11@pci0:0:22:1: class=0x078000 card=0x062b15d9 chip=0x1d3b8086 none12@pci0:0:22:3: class=0x070002 card=0x062b15d9 chip=0x1d3d8086 none13@pci0:0:31:3: class=0x0c0500 card=0x062b15d9 chip=0x1d228086 none14@pci0:0:31:6: class=0x118000 card=0x062b15d9 chip=0x1d248086 none15@pci0:8:0:0: class=0x010700 card=0x062b15d9 chip=0x1d6b8086 none16@pci0:255:8:0: class=0x088000 card=0x062b15d9 chip=0x3c808086 none17@pci0:255:8:3: class=0x088000 card=0x062b15d9 chip=0x3c838086 none18@pci0:255:8:4: class=0x088000 card=0x062b15d9 chip=0x3c848086 none19@pci0:255:9:0: class=0x088000 card=0x062b15d9 chip=0x3c908086 none20@pci0:255:9:3: class=0x088000 card=0x062b15d9 chip=0x3c938086 none21@pci0:255:9:4: class=0x088000 card=0x062b15d9 chip=0x3c948086 none22@pci0:255:10:0: class=0x088000 card=0x062b15d9 chip=0x3cc08086 none23@pci0:255:10:1: class=0x088000 card=0x062b15d9 chip=0x3cc18086 none24@pci0:255:10:2: class=0x088000 card=0x062b15d9 chip=0x3cc28086 none25@pci0:255:10:3: class=0x088000 card=0x062b15d9 chip=0x3cd08086 none26@pci0:255:11:0: class=0x088000 card=0x062b15d9 chip=0x3ce08086 none27@pci0:255:11:3: class=0x088000 card=0x062b15d9 chip=0x3ce38086 none28@pci0:255:12:0: class=0x088000 card=0x15d9 chip=0x3ce88086 none29@pci0:255:12:1: class=0x088000 card=0x15d9 chip=0x3ce88086 none30@pci0:255:12:2: class=0x088000 card=0x15d9 chip=0x3ce88086 none31@pci0:255:12:6: class=0x088000 card=0x15d9 chip=0x3cf48086 none32@pci0:255:12:7: class=0x088000 card=0x15d9 chip=0x3cf68086 none33@pci0:255:13:0: class=0x088000 card=0x15d9 chip=0x3ce88086 none34@pci0:255:13:1: class=0x088000 card=0x15d9 chip=0x3ce88086 none35@pci0:255:13:2: class=0x088000 card=0x15d9 chip=0x3ce88086 none36@pci0:255:13:6: class=0x088000 card=0x15d9 chip=0x3cf58086 none37@pci0:255:14:0: class=0x088000 card=0x062b15d9 chip=0x3ca08086 none38@pci0:255:14:1: class=0x110100 card=0x062b15d9 chip=0x3c468086 none39@pci0:255:15:0: class=0x088000 card=0x062b15d9 chip=0x3ca88086 none40@pci0:255:15:1: class=0x088000 card=0x062b15d9 chip=0x3c718086 none41@pci0:255:15:2: class=0x088000 card=0x062b15d9 chip=0x3caa8086 none42@pci0:255:15:3: class=0x088000 card=0x062b15d9 chip=0x3cab8086 none43@pci0:255:15:4: class=0x088000 card=0x062b15d9 chip=0x3cac8086 none44@pci0:255:15:5: class=0x088000 card=0x062b15d9 chip=0x3cad8086 none45@pci0:255:15:6: class=0x088000 card=0x062b15d9 chip=0x3cae8086 none46@pci0:255:16:0: class=0x088000 card=0x062b15d9 chip=0x3cb08086 none47@pci0:255:16:1: class=0x088000 card=0x062b15d9 chip=0x3cb18086 none48@pci0:255:16:2: class=0x088000 card=0x062b15d9 chip=0x3cb28086 none49@pci0:255:16:3: class=0x088000 card=0x062b15d9 chip=0x3cb38086 none50@pci0:255:16:4: class=0x088000 card=0x062b15d9 chip=0x3cb48086 none51@pci0:255:16:5: class=0x088000 card=0x062b15d9 chip=0x3cb58086 none52@pci0:255:16:6: class=0x088000 card=0x062b15d9 chip=0x3cb68086 none53@pci0:255:16:7: class=0x088000 card=0x062b15d9 chip=0x3cb78086 none54@pci0:255:17:0: class=0x088000 card=0x062b15d9 chip=0x3cb88086 none55@pci0:255:19:0: class=0x088000 card=0x062b15d9 chip=0x3ce48086 none56@pci0:255:19:1: class=0x110100 card=0x062b15d9 chip=0x3c438086 none57@pci0:255:19:4: class=0x110100 card=0x062b15d9 chip=0x3ce68086 none58@pci0:255:19:5: class=0x110100 card=0x062b15d9 chip=0x3c448086 none59@pci0:255:19:6: class=0x088000 card=0x062b15d9 chip=0x3c458086 These would seem to correspond to Uncore feature, namely, as per [0]: 0x3C20 - 0x3C3F: IO Features (QDDMA, APIC, Intel VT, RAS, Intel TXT) 0x3C40 - 0x3C5F: Performance Monitors 0x3C60 - 0x3C7F: DFX 0x3C80 - 0x3C9F: Intel QuickPath Interconnect 0x3CA0 - 0x3CBF: Home Agent/Memory Controller 0x3CC0 - 0x3CDF: Power Management 0x3CE0 - 0x3CFF: Cbo/Ring Even if it would not seem to be critical features, what will be the consequence of having those features currently unhandled ? Thanks, - Arnaud [0]: http://www.intel.vn/content/dam/www/public/us/en/documents/datasheets/xeon-e5-1600-2600-vol-2-datasheet.pdf
Re: [RFT][patch] Scheduling for HTT and not only
Hi, 2012/4/9 Alexander Motin m...@freebsd.org: [...] I have strong feeling that while this test may be interesting for profiling, it's own results in first place depend not from how fast scheduler is, but from the pipes capacity and other alike things. Can somebody hint me what except pipe capacity and context switch to unblocked receiver prevents sender from sending all data in batch and then receiver from receiving them all in batch? If different OSes have different policies there, I think results could be incomparable. Let me disagree on your conclusion. If OS A does a task in X seconds, and OS B does the same task in Y seconds, if Y X, then OS B is just not performing good enough. Internal implementation's difference for the task can not be waived as an excuse for result's comparability. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFT][patch] Scheduling for HTT and not only
Hi, On Tue, Apr 10, 2012 at 1:53 PM, Alexander Motin m...@freebsd.org wrote: On 04/10/12 20:18, Alexander Motin wrote: On 04/10/12 19:58, Arnaud Lacombe wrote: 2012/4/9 Alexander Motinm...@freebsd.org: [...] I have strong feeling that while this test may be interesting for profiling, it's own results in first place depend not from how fast scheduler is, but from the pipes capacity and other alike things. Can somebody hint me what except pipe capacity and context switch to unblocked receiver prevents sender from sending all data in batch and then receiver from receiving them all in batch? If different OSes have different policies there, I think results could be incomparable. Let me disagree on your conclusion. If OS A does a task in X seconds, and OS B does the same task in Y seconds, if Y X, then OS B is just not performing good enough. Internal implementation's difference for the task can not be waived as an excuse for result's comparability. Sure, numbers are always numbers, but the question is what are they showing? Understanding of the test results is even more important for purely synthetic tests like this. Especially when one test run gives 25 seconds, while another gives 50. This test is not completely clear to me and that is what I've told. Small illustration to my point. Simple scheduler tuning affects thread preemption policy and changes this test results in three times: mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 9.568 mav@test:/test/hackbench# sysctl kern.sched.interact=0 kern.sched.interact: 30 - 0 mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 5.163 mav@test:/test/hackbench# sysctl kern.sched.interact=100 kern.sched.interact: 0 - 100 mav@test:/test/hackbench# ./hackbench 30 process 1000 Running with 30*40 (== 1200) tasks. Time: 3.190 I think it affects balance between pipe latency and bandwidth, while test measures only the last. It is clear that conclusion from these numbers depends on what do we want to have. I don't really care on this point, I'm only testing default values, or more precisely, whatever developers though default values would be good. Btw, you are testing 3 differents configuration. Different results are expected. What worries me more is the rather the huge instability on the *same* configuration, say on a pipe/thread/70 groups/600 iterations run, where results range from 2.7s[0] to 7.4s, or a socket/thread/20 groups/1400 iterations run, where results range from 2.4s to 4.5s. - Arnaud [0]: numbers extracted from a recent run of 9.0-RELEASE on a Xeon E5-1650 platform. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Scheduler + IPC performance on FreeBSD 7.4, 8.2, 9.0 and -CURRENT
Hi, On Fri, Apr 6, 2012 at 10:24 AM, Florian Smeets f...@freebsd.org wrote: On 05.04.12 20:03, Arnaud Lacombe wrote: Hi folks, Hi, Over the past months, I ran on a couple of unused box the `hackbench'[HACKBENCH] benchmark used by the Linux folks for tracking down various kind of regression/improvement. `hackbench' is a scheduler + IPC test (socket xor pipe). It creates producers/consumers groups and let a variable quantity of small messages flow happily. Producers and consumers are either processes xor threads. [Lots of likely very interesting and valuable data.] Q4: So, how can I get all the graph ? R4: All you need is git, a posix shell, a couple of utility (find, sort, ...), a recent gnuplot, and a ruby interpreter. Can you give us some hints on *how* to get the results? I checked the repo out but it's not immediately obvious what to do and how to get the graphs, as staring at thousands of numbers in lots of different files isn't exactly practical. To just get all the graph, merge the runs/* branch you want, and just run the `results.sh' script: # sh results.sh To gather result, build `hackbench': # eval $(sed '/#gcc/!d; s/.//' hackbench.c) then, reboot in single mode, mount / read-write, adjust whatever you have to adjust and run the script: # sh hackbench.sh [light|medium|heavy] $(pwd)/hackbench this will run a complete iterations over all the possible tunables and gives you a `results.yml' that you can feed to the previous script. - Arnaud Thanks, Florian ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Scheduler + IPC performance on FreeBSD 7.4, 8.2, 9.0 and -CURRENT
Hi, On Fri, Apr 6, 2012 at 10:58 AM, Attilio Rao atti...@freebsd.org wrote: Il 05 aprile 2012 19:03, Arnaud Lacombe lacom...@gmail.com ha scritto: Hi folks, Over the past months, I ran on a couple of unused box the `hackbench'[HACKBENCH] benchmark used by the Linux folks for tracking down various kind of regression/improvement. `hackbench' is a scheduler + IPC test (socket xor pipe). It creates producers/consumers groups and let a variable quantity of small messages flow happily. Producers and consumers are either processes xor threads. Tested platforms were - Atom D510, Intel, (incomplete) - Core 2 Quad Q9560, Intel - Soekris net5501, AMD (incomplete) - Xeon E5645, Intel (incomplete) - Xeon E5620 (dual package), Intel - Xeon E5-1650 (pending completion) - Vortex86, DMP Tested kernel were: - FreeBSD 7.4-RELEASE - FreeBSD 8.2-RELEASE - FreeBSD 9.0-RC3 and FreeBSD 9.0-RELEASE - FreeBSD 10-CURRENT as of r231573 Which means you run 10-CURRENT with all the kernel debugging options on and MALLOC_DEBUG on? I already answered that question. Namely: note: rule [I] is alleviated for -CURRENT kernels, which were built with the same alteration made to GENERIC during the CURRENT-RELEASE transition (ie. WITNESS and a couple of other option disabled). this translates into the following patch (for amd64): diff --git a/sys/amd64/conf/GENERIC b/sys/amd64/conf/GENERIC index 8db8e27..9d61f25 100644 --- a/sys/amd64/conf/GENERIC +++ b/sys/amd64/conf/GENERIC @@ -67,20 +67,6 @@ options MAC # TrustedBSD MAC Framework #options KDTRACE_HOOKS # Kernel DTrace hooks optionsINCLUDE_CONFIG_FILE # Include this file in kernel -# Debugging support. Always need this: -optionsKDB # Enable kernel debugger support. -# For minimum debugger support (stable branch) use: -#options KDB_TRACE # Print a stack trace for a panic. -# For full debugger support use this instead: -optionsDDB # Support DDB. -optionsGDB # Support remote GDB. -optionsDEADLKRES # Enable the deadlock resolver -optionsINVARIANTS # Enable calls of extra sanity checking -optionsINVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS -optionsWITNESS # Enable checks to detect deadlocks and cycles -optionsWITNESS_SKIPSPIN# Don't run witness on spinlocks for speed -optionsMALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones - Arnaud Attilio -- Peace can only be achieved by understanding - A. Einstein ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Scheduler + IPC performance on FreeBSD 7.4, 8.2, 9.0 and -CURRENT
Hi, On Fri, Apr 6, 2012 at 1:55 PM, Attilio Rao atti...@freebsd.org wrote: Il 06 aprile 2012 18:54, Arnaud Lacombe lacom...@gmail.com ha scritto: Hi, On Fri, Apr 6, 2012 at 10:58 AM, Attilio Rao atti...@freebsd.org wrote: Il 05 aprile 2012 19:03, Arnaud Lacombe lacom...@gmail.com ha scritto: Hi folks, Over the past months, I ran on a couple of unused box the `hackbench'[HACKBENCH] benchmark used by the Linux folks for tracking down various kind of regression/improvement. `hackbench' is a scheduler + IPC test (socket xor pipe). It creates producers/consumers groups and let a variable quantity of small messages flow happily. Producers and consumers are either processes xor threads. Tested platforms were - Atom D510, Intel, (incomplete) - Core 2 Quad Q9560, Intel - Soekris net5501, AMD (incomplete) - Xeon E5645, Intel (incomplete) - Xeon E5620 (dual package), Intel - Xeon E5-1650 (pending completion) - Vortex86, DMP Tested kernel were: - FreeBSD 7.4-RELEASE - FreeBSD 8.2-RELEASE - FreeBSD 9.0-RC3 and FreeBSD 9.0-RELEASE - FreeBSD 10-CURRENT as of r231573 Which means you run 10-CURRENT with all the kernel debugging options on and MALLOC_DEBUG on? I already answered that question. Namely: note: rule [I] is alleviated for -CURRENT kernels, which were built with the same alteration made to GENERIC during the CURRENT-RELEASE transition (ie. WITNESS and a couple of other option disabled). this translates into the following patch (for amd64): Did you enable MALLOC_PRODUCTION and rebuilt libc? Userland originates from FreeBSD 7.4-RELEASE and was not changed for any of the tests, which are exclusively focused on the kernel. Doing otherwise would mean changing too many variables. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Scheduler + IPC performance on FreeBSD 7.4, 8.2, 9.0 and -CURRENT
Hi folks, Over the past months, I ran on a couple of unused box the `hackbench'[HACKBENCH] benchmark used by the Linux folks for tracking down various kind of regression/improvement. `hackbench' is a scheduler + IPC test (socket xor pipe). It creates producers/consumers groups and let a variable quantity of small messages flow happily. Producers and consumers are either processes xor threads. Tested platforms were - Atom D510, Intel, (incomplete) - Core 2 Quad Q9560, Intel - Soekris net5501, AMD (incomplete) - Xeon E5645, Intel (incomplete) - Xeon E5620 (dual package), Intel - Xeon E5-1650 (pending completion) - Vortex86, DMP Tested kernel were: - FreeBSD 7.4-RELEASE - FreeBSD 8.2-RELEASE - FreeBSD 9.0-RC3 and FreeBSD 9.0-RELEASE - FreeBSD 10-CURRENT as of r231573 on the following architecture: - amd64 (if supported, incomplete) - i386 1) DISCLAMER Let me start by pointing out something important: [I] I am _not_ interested in testing released version FOO with feature BAR enabled, if enabling BAR require a kernel rebuild. All tests for release kernels were made for the kernel shipped officially, it is the developers responsability to decide whether or not enable a feature by default, not mine. If option BAR gives a 20% performance improvement, enable it, don't complain the test to be 20% slower. [II] I am _not_ interested in altering any hints, tunables or sysctl, unless they prevent the execution of the test. The exception added to the above rule is due to limitation introduced by `kern.threads.max_threads_per_proc' and `kern.ipc.maxpipekva'. Those were respectively set to 8192 and between 16M/64M. note: rule [I] is alleviated for -CURRENT kernels, which were built with the same alteration made to GENERIC during the CURRENT-RELEASE transition (ie. WITNESS and a couple of other option disabled). 2) Tests description `hackbench' has the following tunable: - IPC to use for messaging, either `pipe' or `socket'. - Threading model, either `thread' or `process' - Number of iteration to run - Number of group to create The tests covered all of these adjustments more or less heavily depending on the platform capability. 3) Scripts Test scripts are available in the `master' branch of the git repository at: https://github.com/lacombar/hackbench in the `hackbench/' directory. 4) Results Full results are available in the `runs/*' branches of the GitHub repository. 5) Quick results summary * UP case FreeBSD 9.0 behaves better than FreeBSD 8.2 in process mode, especially with sockets. Results are comparable with thread. 9.0-RC3 shows a 10% hits in thread/socket mode on the LX800, this will need confirmation. Linux is stable and scales linearly in all situation. It is only beaten by FreeBSD 8.2-RELEASE with thread/socket. * MP case These is a pretty bad regression with FreeBSD 9.0 in thread/pipe mode, which scale almost in O(N^2), ending up in way worse performance than FreeBSD 7.4 or 8.2 on the Core 2 Quad. Beside that, it is really difficult to draw a general trends, ie. whether FreeBSD 9.0 behaves better than FreeBSD 8.2, or the other way around. Pretty much all situation arises, FreeBSD 9.0 can beat FreeBSD 8.2 on some workload, behave the same, or be beaten on others. None really scales regularly either. Pretty much every runs shows thresholds where scheduling decision change and/or became erratic. 6) Anticipated question and remarks Q1: You should truly enable kick-ass feature BAZ in the kernel. R1: I'm lazy. Do your job as a developer to integrate the feature. If it should be the default, make it the default. Q2: You should set `kern.vm.whatever' to 42, or enable feature BAZ in the kernel, to get full performance from the Warp engine on Constellation-class starship. R2: Would you ask Lt. Worf to re-aligh plasma injectors or would ask Lt. Commander La Forge to plan an assault, seriously ? Q3: You built the Linux kernel, why can't you rebuild FreeBSD's ? For a couple of reason: - the Linux kernel does not provide binary release per-se. - the Linux kernel was not the focus of the tests, but merely a comparative of what others-can-do. - I did not tweak the Linux kernel configuration. The kernels configuration tested derived from the `defconfig', with very few amendment[0], mostly about hardware support not enabled by default Q3: Could you post all the graph ? R3: I could, but there is really tons of them, so posting a subset of them would be subjective, all the materials is available on the git repository. Q4: So, how can I get all the graph ? R4: All you need is git, a posix shell, a couple of utility (find, sort, ...), a recent gnuplot, and a ruby interpreter. Comments and suggestions will be greatly appreciated. - Arnaud [HACKBENCH]: http://people.redhat.com/mingo/cfs-scheduler/tools/hackbench.c [0]: the exact list is: # CONFIG_KERNEL_GZIP is not set CONFIG_KERNEL_XZ=y CONFIG_IKCONFIG=y CONFIG_IKCONFIG_PROC=y # CONFIG_MODULES is not set
Re: [RFT][patch] Scheduling for HTT and not only
Hi, [Sorry for the delay, I got a bit sidetrack'ed...] 2012/2/17 Alexander Motin m...@freebsd.org: On 17.02.2012 18:53, Arnaud Lacombe wrote: On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motinm...@freebsd.org wrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) is there a place where all the patches are available ? All my scheduler patches are cumulative, so all you need is only the last mentioned here sched.htt40.patch. You may want to have a look to the result I collected in the `runs/freebsd-experiments' branch of: https://github.com/lacombar/hackbench/ and compare them with vanilla FreeBSD 9.0 and -CURRENT results available in `runs/freebsd'. On the dual package platform, your patch is not a definite win. But in some cases, especially for multi-socket systems, to let it show its best, you may want to apply additional patch from avg@ to better detect CPU topology: https://gitorious.org/~avg/freebsd/avgbsd/commit/6bca4a2e4854ea3fc275946a023db65c483cb9dd test I conducted specifically for this patch did not showed much improvement... - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [CFT] modular kernel config
Hi, 2012/2/27 Łukasz Wąsikowski luk...@wasikowski.net: W dniu 2012-02-22 23:31, Bjoern A. Zeeb pisze: You cannot ship that on by default for non-tecnical reasons in a kernel. Please do not commit a kernel config that can be booted (no LINT cannot be booted) with these on without consulting appropriate hats upfront. - ALTQ - SW_WATCHDOG - QUOTA - IPSTEALTH (disabled in loader.conf) - IPFIREWALL_FORWARD (touches every packet, power users which need a bigger PPS but not this feature can recompile the kernel, discussed with julian@) - FLOWTABLE (disabled in loader.conf) Which is not the same as it's not 100% disabled and will still allocate memory. FLOWTABLE on 8.x crashed BGP routers (kern/144917). no crash dump, no backtrace, no follow-up whatsoever after 1 year and 2 years, what's your points ? You could really have chosen a better PR to back up your argument... - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [CFT] modular kernel config
Hi, 2012/2/27 Steve Wills swi...@freebsd.org: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/27/12 10:53, Łukasz Wąsikowski wrote: W dniu 2012-02-22 23:31, Bjoern A. Zeeb pisze: You cannot ship that on by default for non-tecnical reasons in a kernel. Please do not commit a kernel config that can be booted (no LINT cannot be booted) with these on without consulting appropriate hats upfront. - ALTQ - SW_WATCHDOG - QUOTA - IPSTEALTH (disabled in loader.conf) - IPFIREWALL_FORWARD (touches every packet, power users which need a bigger PPS but not this feature can recompile the kernel, discussed with julian@) - FLOWTABLE (disabled in loader.conf) Which is not the same as it's not 100% disabled and will still allocate memory. FLOWTABLE on 8.x crashed BGP routers (kern/144917). I don't know if it is fixed by now, but this kind of potential problematic features should not be enabled by default. Agree, I've run into problems with FLOWTABLE (with just the features that were enabled by default in 8.0) when routers changed MAC addresses. As far as I understand it, FLOWTABLE is both broken and abandoned (but if I'm wrong, please let me know). You will sure go really far with this kind of It is broken ? Let's not fix it and disable it instead mentality, even more when coming from a committer. As long as there will be these kind of comments around here, FreeBSD will deserve nothing but to keep dying piece by piece, and it will be deserved. - Arnaud So, IMHO, not only should it not be enabled by default, but given that it was disabled complete in 8.x after 8.0 (too lazy to look at exactly when right now), I think it shouldn't even be included, since that might encourage users to try it out only to encounter problems with it. Steve -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.18 (FreeBSD) iQEcBAEBAgAGBQJPTD/nAAoJEPXPYrMgexuhvWAH/iPa0N32GJXdyL7OdqFNNuEN R/Z0tOqTCCmAm4WsbAbN3m5poBKNctRtQxG40XoqmvZAWlomwYCwpS2xgCX60NZO XhspUEpQ7cHQpt6ZOW12x3G6FZJ4DzFX0+mDPYxE/7YmwtsjZFeTFGVEPezeKQwr Khar5jWYqETmM1+mFvAFXnfTUiBwnqErDfYmHQAE93wKQd9CyzrFmDfooNTiMUB6 yW+piexN/SzXz6nftQesJbFOWUW6fvhxe9TYcK8+b9zCo2GxPuUrRV60PKQX0apd nlKWtj49znLVzmpTYboQnvmqmk+yik8wL2wszUaZ4jAjieCjWOhJwCWOvkQ9yIg= =SBbK -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [CFT] modular kernel config
Hi, 2012/2/28 Łukasz Wąsikowski luk...@wasikowski.net: W dniu 2012-02-28 19:55, Arnaud Lacombe pisze: FLOWTABLE on 8.x crashed BGP routers (kern/144917). no crash dump, no backtrace, no follow-up whatsoever after 1 year and 2 years, what's your points ? You could really have chosen a better PR to back up your argument... Sorry, but I don't want to bug trace this issue, simply because lack of time, resources and interest in this feature. I've run into this bug on production box, went through hell because of it and turned off flowtable which I do not use and not need. If this problem is still alive (it might be, the PR I've mentioned is still open) then it's not a good idea to turn on this feature by default. If you're interested in using this feature then feel free to debug and test. Give me a deterministic way to reproduce the issue and I will. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFT][patch] Scheduling for HTT and not only
Hi, On Fri, Feb 17, 2012 at 11:29 AM, Alexander Motin m...@freebsd.org wrote: On 02/15/12 21:54, Jeff Roberson wrote: On Wed, 15 Feb 2012, Alexander Motin wrote: I've decided to stop those cache black magic practices and focus on things that really exist in this world -- SMT and CPU load. I've dropped most of cache related things from the patch and made the rest of things more strict and predictable: http://people.freebsd.org/~mav/sched.htt34.patch This looks great. I think there is value in considering the other approach further but I would like to do this part first. It would be nice to also add priority as a greater influence in the load balancing as well. I haven't got good idea yet about balancing priorities, but I've rewritten balancer itself. As soon as sched_lowest() / sched_highest() are more intelligent now, they allowed to remove topology traversing from the balancer itself. That should fix double-swapping problem, allow to keep some affinity while moving threads and make balancing more fair. I did number of tests running 4, 8, 9 and 16 CPU-bound threads on 8 CPUs. With 4, 8 and 16 threads everything is stationary as it should. With 9 threads I see regular and random load move between all 8 CPUs. Measurements on 5 minutes run show deviation of only about 5 seconds. It is the same deviation as I see caused by only scheduling of 16 threads on 8 cores without any balancing needed at all. So I believe this code works as it should. Here is the patch: http://people.freebsd.org/~mav/sched.htt40.patch I plan this to be a final patch of this series (more to come :)) and if there will be no problems or objections, I am going to commit it (except some debugging KTRs) in about ten days. So now it's a good time for reviews and testing. :) is there a place where all the patches are available ? I intend to run some tests on a 1x2x2 (atom D510), 1x4x1 (core-2 quad), and eventually a 2x8x2 platforms, against r231573. Results should hopefully be available by the end of the week-end/middle of next week[0]. - Arnaud [0]: the D510 will likely be testing a couple of Linux kernel over the week-end, and a FreeBSD run takes about 2.5 days to complete. -- Alexander Motin ___ freebsd-hack...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: stable/9 still looking for packages at 9-current
Hi, On Mon, Jan 9, 2012 at 9:37 AM, Mark Linimon lini...@lonesome.com wrote: On 9. Jan 2012, at 01:04 , Arnaud Lacombe wrote: So you are saying that FreeBSD is currently providing on ftp://ftp.freebsd.org/pub images tagged as being 9.0 RELEASE (with checksum provided), in a `releases' directory, which are not actually release images per-se ? Excellent! You've shown the ability to understand flat, declarative, sentences that have no qualifying phrases. FWIW, this was more a sarcastic sentence pointing out that FreeBSD is currently officially distributing non-released build in a directory which might leads users to consider this is the official release, thus misleading them. 9.0 will be *released* when and only when the official, signed, email goes out. Everything up until that point is preparation. ok, I'm a stupid lazy user (obviously)... While browsing the ftp, I see 9.0 ISOs in a `releases' directory. Do you expect me to consult freebsd-announce@, verify the signature of the announce, the hash of the ISOs, etc. to consider that 9.0 has been released ? No, I see 9.0 ISOs in a `releases' directory, I assume it has been released, whatever your spreading process is. Btw, none of the CHECKSUMS files are signed on the FTP. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: stable/9 still looking for packages at 9-current
Hi, On Mon, Jan 9, 2012 at 1:40 PM, Freddie Cash fjwc...@gmail.com wrote: On Mon, Jan 9, 2012 at 10:27 AM, Chris Rees utis...@gmail.com wrote: On 9 January 2012 18:16, Arnaud Lacombe lacom...@gmail.com wrote: ok, I'm a stupid lazy user (obviously)... While browsing the ftp, I see 9.0 ISOs in a `releases' directory. Do you expect me to consult freebsd-announce@, verify the signature of the announce, the hash of the ISOs, etc. to consider that 9.0 has been released ? No, I see 9.0 ISOs in a `releases' directory, I assume it has been released, whatever your spreading process is. Btw, none of the CHECKSUMS files are signed on the FTP. Have you checked the website? The latest supported release is clearly specified, right in the middle of the home page. Please don't tell me you'd look in ftp before checking the website. I think you're just looking to nitpick. And, which is worse: 1. tag the release branch, build the ISOs, upload to main FTP server, wait for the mirrors (FTP, CVS, SVN) to sync, then make the official announcement which includes a few days/weeks where the release is available but not official; or 2. tag the release branch, build the ISOs, upload to main FTP server, make the official announcement, user goes to their favourite/closest mirror, and can't access the release since it hasn't synced yet 3. tag the release[0], build the ISOs, upload then to main FTP and mirrors but restrict visibility[0], announce the release, make the build visible everywhere. At worse, fall-back on #2 with peer-to-peer distribution. If some steps in that process do not exist, create them. - Arnaud I think people would complain a hell of a lot more about 2 than they currently do about 1. Yes, people upgrading via source will see X.Y-RELEASE before it's officially announced on the website/mailing lists. Yes, people browsing ftp.freebsd.org will see X.Y-RELEASE ISOs before it's officially announced. Yes, some users will get confused by seeing X.Y-RELEASE available before the official annoucements. But, that's a lot better than making an annoucement and having users unable to use it since it's not available on their local mirrors. What's annoying, though, is that we have to go through this with every ... single ... minor ... release. It's not a hard concept, yet every time there's a new release, people get confused by it. Is there something that could be done to make it more streamlined/smoother? Maybe, maybe not. Depends. You'd have to want to join the RE team to find out more about the current release/mirror infrastructure. :) And then be willing to put in the time/effort to improve it. :D Does all of Arnaud's complaining and nit-picking constitute a request to volunteer to fix things? ;) -- Freddie Cash fjwc...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: stable/9 still looking for packages at 9-current
Hi, On Mon, Jan 9, 2012 at 1:27 PM, Chris Rees utis...@gmail.com wrote: On 9 January 2012 18:16, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Mon, Jan 9, 2012 at 9:37 AM, Mark Linimon lini...@lonesome.com wrote: On 9. Jan 2012, at 01:04 , Arnaud Lacombe wrote: So you are saying that FreeBSD is currently providing on ftp://ftp.freebsd.org/pub images tagged as being 9.0 RELEASE (with checksum provided), in a `releases' directory, which are not actually release images per-se ? Excellent! You've shown the ability to understand flat, declarative, sentences that have no qualifying phrases. FWIW, this was more a sarcastic sentence pointing out that FreeBSD is currently officially distributing non-released build in a directory which might leads users to consider this is the official release, thus misleading them. So, a pointless email. as is linimon@'s. 9.0 will be *released* when and only when the official, signed, email goes out. Everything up until that point is preparation. ok, I'm a stupid lazy user (obviously)... While browsing the ftp, I see 9.0 ISOs in a `releases' directory. Do you expect me to consult freebsd-announce@, verify the signature of the announce, the hash of the ISOs, etc. to consider that 9.0 has been released ? No, I see 9.0 ISOs in a `releases' directory, I assume it has been released, whatever your spreading process is. Btw, none of the CHECKSUMS files are signed on the FTP. Have you checked the website? The latest supported release is clearly specified, right in the middle of the home page. Please don't tell me you'd look in ftp before checking the website. I think you're just looking to nitpick. I did look the ftp before the website. It is an irrelevant source of information as I assume none of that stuff to be up-to-date. re@ has an unimpressive track record about information update. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: stable/9 still looking for packages at 9-current
Hi, On Thu, Jan 5, 2012 at 5:12 PM, Peter fb...@peterk.org wrote: Hello, Installed 9-RELEASE amd64, [...] Has 9.0 been released ? I cannot find any announcement, especially on freebsd-announce@, [9.0TODO] has not been updated, there is no ISO image in [0], but there is in [3], dated from Jan 5th 2012 and `origin/releng/9.0' appeared in the . Thanks, - Arnaud [9.0TODO]: http://wiki.freebsd.org/Releng/9.0TODO [0]: ftp://ftp.freebsd.org/pub/FreeBSD/ISO-IMAGES-amd64 [1]: ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/9.0/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: stable/9 still looking for packages at 9-current
Hi, On Sun, Jan 8, 2012 at 7:30 PM, Mark Linimon lini...@lonesome.com wrote: On Sun, Jan 08, 2012 at 07:26:47PM -0500, Eitan Adler wrote: On Sun, Jan 8, 2012 at 7:22 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Thu, Jan 5, 2012 at 5:12 PM, Peter fb...@peterk.org wrote: Hello, Installed 9-RELEASE amd64, [...] Has 9.0 been released ? 9.0 will, and only will, be released when an announcement is made on freebsd-annou...@freebsd.org, containing such things as verified checksums. So you are saying that FreeBSD is currently providing on ftp://ftp.freebsd.org/pub images tagged as being 9.0 RELEASE (with checksum provided), in a `releases' directory, which are not actually release images per-se ? - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
Hi, On Wed, Jan 4, 2012 at 4:42 PM, Chuck Swiger cswi...@mac.com wrote: On Jan 4, 2012, at 1:03 PM, Dan The Man wrote: However, I'm not convinced that it is useful to do this. At some point, you are better off timing out and retrying via exponential backoff than you are queuing hundreds of thousands of connections in the hopes that they will eventually be serviced by something sometime considerably later. I agree completely, in practical application this makes sense, but why should the OS dictate not being able to temporarily set that setting higher in order to fully benchmark the application at 100k+ in the listen queue if the developer so chooses? I think that alone should be a good reason, to make freebsd developer friendly. The job of the OS is to manage resources on behalf of the users and processes using the system. No. The job of the OS is to service the user with the resource available, not constrict the user within some arbitrary predefined wall when there is still plenty of room available. If resource become scarce, then take action. Some developers feel that VM means that the system should always claim have more memory available, but always saying yes isn't managing resources. I'd rather have the OS return a null pointer and set ENOMEM when someone tries to malloc() more memory than the system (including swap, VM overcommit, etc) has, and I expect developers to code well enough to handle malloc() failures. this is merely a policy issue, not yours to impose. Setting the listen queue to an arbitrarily high value isn't useful, and developers would be better advised to pay attention to best practices in the face of a massive connection backlog. Stress-testing isn't about best practice. It is about shaking enough the system to highlight weak point. - Arnaud Regards, -- -Chuck ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sysctl kern.ipc.somaxconn limit 65535 why?
Hi, On Wed, Jan 4, 2012 at 3:22 PM, Dan The Man d...@sunsaturn.com wrote: sunsaturn:~# sysctl -w kern.ipc.somaxconn=20 kern.ipc.somaxconn: 4096 sysctl: kern.ipc.somaxconn: Invalid argument sunsaturn:~# sysctl -w kern.ipc.somaxconn=65536 kern.ipc.somaxconn: 4096 sysctl: kern.ipc.somaxconn: Invalid argument sunsaturn:~# sysctl -w kern.ipc.somaxconn=65535 kern.ipc.somaxconn: 4096 - 65535 sunsaturn:~# Trying to stress test a framework here that tosses 100k of connections into a listen queue before doing anything, I realize I'll have to use multiple local IPs get get around port limitations, but why is this backlog using a limit? This is an arbitrary implementation limitation. A bit of code archaeology reveals that the explicit limitation was introduced in 1,226 of kern/uipc_socket.c[0]. Quoting the commit log: glebus@ - Convert so_qlen, so_incqlen, so_qlimit fields of struct socket from short to unsigned short. - Add SYSCTL_PROC() around somaxconn, not accepting values 1 or U_SHRTMAX. Before this change setting somaxconn to smth above 32767 and calling listen(fd, -1) lead to a socket, which doesn't accept connections at all. Reviewed by:rwatson Reported by:Igor Sysoev so the limited width of some `struct socket' fields enforces this limitation. Now, the reason for `so_qlen', `so_incqlen', `so_qlimit' to be so limited might be lost in the arcane of time. After all 640K ought to be enough for anyone, doesn't it ? - Arnaud [0]: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/uipc_socket.c#rev1.226 Dan. -- Dan The Man CTO/ Senior System Administrator Websites, Domains and Everything else http://www.SunSaturn.com Email: d...@sunsaturn.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, On Fri, Dec 16, 2011 at 12:16 PM, matt...@phoronix.com wrote: Thanks. My request for the person documenting the tunings also runs the benchmark to ensure expected behaviour. Why should you have to tune anything ? Did you tune the Oracle Server install ? If not, you should not have to tune the FreeBSD install, that wouldn't be fair. If you tune FreeBSD, you should tune the Oracle Server install too. It is pretty easy to win at least 30% in performance for certain workload by choosing the right kernel configuration. - Arnaud The installation, execution and comparison against the benchmarks in the article is fairly simple. Note that some tuning may not be relevant or recommended (ie: some of the fs benchmarks are sensitive to barriers and other synchronous operations). I'd recommend bowing out of a benchmark with a 'we're going to be slower since the default configuration is this way for the following reason' if this is the case. Thanks 'someone'. Matthew Dec 16, 2011 8:46 AM, Adrian Chadd adr...@freebsd.org wrote: Can someone please write up a nice, concise blog post somewhere outlining all of this? Extra bonus points if it's a blog that is picked up by blogs.freebsdish.org and/or some of the other BSD sites. Guys/girls/fuzzy things - this is 2011; people look at shiny blog sites with graphs rather than mailing lists. Sorry, we lost that battle. :) Adrian ___ freebsd-performa...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to freebsd-performance-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, [resend on the ml, my bad] On Fri, Dec 16, 2011 at 5:54 AM, Attilio Rao atti...@freebsd.org wrote: 2011/12/16 Arnaud Lacombe lacom...@gmail.com: Hi, On Thu, Dec 15, 2011 at 2:32 AM, O. Hartmann ohart...@zedat.fu-berlin.de wrote: Just saw this shot benchmark on Phoronix dot com today: http://www.phoronix.com/scan.php?page=news_itempx=MTAyNzA it might be worth highlighting that despite Oracle Linux 6.1 Server is using a kernel + compiler almost 2 years old, it still manages to out-perform the bleeding edge FreeBSD :-) Now, from what I've read so far in this thread, it seems that a lot of people are still in abnegation... my 0.2c, - Arnaud Said by someone which really thinks passing __FILE__ and __LINE__ to kernel function is going to give a mesaurable performance penalty is really hilarious however :) You are right, the rest of the kernel's subsystem are so sluggish, fragile and half baked that this would barely improve anything... That will be my last word in this thread. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server
Hi, On Thu, Dec 15, 2011 at 2:32 AM, O. Hartmann ohart...@zedat.fu-berlin.de wrote: Just saw this shot benchmark on Phoronix dot com today: http://www.phoronix.com/scan.php?page=news_itempx=MTAyNzA it might be worth highlighting that despite Oracle Linux 6.1 Server is using a kernel + compiler almost 2 years old, it still manages to out-perform the bleeding edge FreeBSD :-) Now, from what I've read so far in this thread, it seems that a lot of people are still in abnegation... my 0.2c, - Arnaud It may be worth to discuss the sad performance of FBSD in some parts of the benchmark. A difference of a factor 10 or 100 is simply far beyond disapointing, it is more than inacceptable and by just reading those benchmarks, I'd like to drop thinking of using FreeBSD even as a backend server in scientific and business environments. In detail, some of the SciMark benches look disappointing. The overall image can't help over the fact that in C-Ray FreeBSD is better performing. From the compiler, I'd like say there couldn't be a drop of more than 10 - 15% in performance - but not 10 or 100 times. I'm just thinking about the discussion of SCHED_ULE and all the saur spots we discussed when I stumbled over the test. Regards, Oliver ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
PAE broken on -current, likely broken on stable/9
Hi, The kernel tree is utterly broken when PAE is enabled, it chokes [non-exclusively] on the following: dev/dpt/dpt_scsi.c:279: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] dev/dpt/dpt_scsi.c:279: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] dev/dpt/dpt_scsi.c:964: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] dev/ida/ida.c:145: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] dev/ida/ida.c:145: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] dev/ida/ida.c:154: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] dev/ida/ida.c:154: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] dev/ida/ida_eisa.c:105: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] dev/ida/ida_eisa.c:106: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] dev/malo/if_malo_pci.c:232: warning: large integer implicitly truncated to unsigned type [-Woverflow] dev/malo/if_malo_pci.c:232: warning: large integer implicitly truncated to unsigned type [-Woverflow] dev/mwl/if_mwl_pci.c:210: warning: large integer implicitly truncated to unsigned type [-Woverflow] dev/mwl/if_mwl_pci.c:210: warning: large integer implicitly truncated to unsigned type [-Woverflow] dev/sym/sym_hipd.c:7922: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] dev/trm/trm.c:648: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] dev/hptmv/entry.c:2635: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] dev/hptmv/entry.c:2765: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] dev/hptmv/entry.c:2970: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] Could re@ please includes, proudly (or not), in its future announces, that PAE is no longer supported ? Thanks, - Arnaud ps: this is just a report, I'm not really expecting anything, any longer, from the FreebSD community. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: PAE broken on -current, likely broken on stable/9
Hi *, [I could have renamed the subject 1001 fancy ways to crash FreeBSD, but I'll avoid :)] On Mon, Dec 5, 2011 at 5:15 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, The kernel tree is utterly broken when PAE is enabled, it chokes [non-exclusively] on the following: After finally having been able to complete a build, the resulting kernel miserably panics on: real memory: 25769803776 (24576 MB) panic: kmem_suballoc: bad status return of 3 This was with the default value of `vm.kmem_size' and `vm.kmem_size_max'. I cannot find a good value for either of them. With 2GB of RAM and 9.0RC2 (the release kernel), 700MB of kmem boots fine. The same and 750MB of kmem chokes, when bringing up userland, on: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xbfc0 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0d4baca stack pointer = 0x28:0xc520f9dc frame pointer = 0x28:0xc520fa14 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags= interrupt enabled, IOPL = 0 current process = 1 (kernel) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xc0a4b027 at kdb_backtrace+0x47 #1 0xc0a185f7 at panic+0x117 #2 0xc0d48a03 at trap_fatal+0x323 #3 0xc0d48abd at trap_pfault+0xad #4 0xc0d49845 at trap+0x465 #5 0xc0d3279c at calltrap+0x6 #6 0xc09e57a0 at exec_map_first_page+0x430 #7 0xc09e61fc at kern_execve+0x58c #8 0xc09e75bc at sys_execve+0x4c #9 0xc09cb372 at start_init+0x292 #10 0xc09ea8d7 at fork_exit+0x97 #11 0xc0d32814 at fork_trampoline+0x8 Uptime: 1s Automatic reboot in 15 seconds - press a key on the console to abort With 12GB of RAM and 700MB of kmem, chokes early on: CPU: QEMU Virtual CPU version 0.14.50 (2660.71-MHz 686-class CPU) Origin = GenuineIntel Id = 0x633 Family = 6 Model = 3 Stepping = 3 Features=0x781abf9FPU,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2 Features2=0x8081SSE3,POPCNT,HV real memory = 12884901888 (12288 MB) panic: kmem_suballoc: bad status return of 3 cpuid = 0 KDB: enter: panic [ thread pid 0 tid 0 ] Stopped at kdb_enter+0x3a: movl$0,kdb_why db bt Tracing pid 0 tid 0 td 0xc068edb0 kdb_enter(c0603b0a,c0603b0a,c061fbb4,c08f6cbc,0,...) at kdb_enter+0x3a panic(c061fbb4,3,0,0,c06c3a54,...) at panic+0x134 kmem_suballoc(c0ba6000,c06c3a54,c06c3a58,90f8000,1,...) at kmem_suballoc+0x85 vm_ksubmap_init(c06c3a4c,0,3,3000,0,...) at vm_ksubmap_init+0xbc cpu_startup(0,8f0020,8f0020,8f,8fb000,...) at cpu_startup+0x27c mi_startup() at mi_startup+0xac begin() at begin+0x2c db Reverting to the default value for `vm.kmem_size' and `vm.kmem_size_max', 4GB (and 6GB) of RAM, with a PAE-enabled -current kernel triggers an infinite loop of: CPU: QEMU Virtual CPU version 0.14.50 (2660.40-MHz 686-class CPU) Origin = GenuineIntel Id = 0x633 Family = 6 Model = 3 Stepping = 3 Features=0x781abf9FPU,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2 Features2=0x8081SSE3,POPCNT,HV real memory = 6442450944 (6144 MB) kernel trap 12 with interrupts disabled kernel trap 12 with interrupts disabled kernel trap 12 with interrupts disabled kernel trap 12 with interrupts disabled [...] kernel trap 12 with interrupts disabled At this point, even FreeBSD 7.1 is better, as it goes at least up until: Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.1-RELEASE-p13 #0: Mon Nov 21 17:23:05 UTC 2011 root@build:/freebsd/conf/PAE Timecounter i8254 frequency 1193182 Hz quality 0 CPU: QEMU Virtual CPU version 0.14.50 (2660.26-MHz 686-class CPU) Origin = GenuineIntel Id = 0x633 Stepping = 3 Features=0x781abf9FPU,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2 Features2=0x8081SSE3,POPCNT,b31 real memory = 16642998272 (15872 MB) avail memory = 15784312832 (15053 MB) It hanged there for a while, I'm not sure if it's because the system is running on a VM with a disk-backed memory or another issue. I killed qemu at this point. 6GB was fine too. Coming back to -current, but now with `vm.kmem_size' and `vm.kmem_size_max' set to 512M, a 12G system boots: CPU: QEMU Virtual CPU version 0.14.50 (2660.39-MHz 686-class CPU) Origin = GenuineIntel Id = 0x633 Family = 6 Model = 3 Stepping = 3 Features=0x781abf9FPU,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2 Features2=0x8081SSE3,POPCNT,HV real memory = 12884901888 (12288 MB) avail memory = 12621688832 (12036 MB) Event timer LAPIC quality 400 ACPI APIC Table: BOCHS BXPCAPIC ioapic0: Changing APIC ID to 1 ioapic0 Version 1.1 irqs 0-23 on motherboard [...] up until right before multi-user, where it just
Re: PAE broken on -current, likely broken on stable/9
Hi, On Mon, Dec 5, 2011 at 11:12 PM, Arnaud Lacombe lacom...@gmail.com wrote: On Mon, Dec 5, 2011 at 5:15 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, The kernel tree is utterly broken when PAE is enabled, it chokes [non-exclusively] on the following: After finally having been able to complete a build, the resulting kernel miserably panics on: real memory: 25769803776 (24576 MB) panic: kmem_suballoc: bad status return of 3 Just for the fun, after having fought for 8h for nearly nothing with FreeBSD, I thought it would be fair to give it a try with Linux, say the latest -rc, that is 3.2-rc3, with a snapshot of the upcoming openwrt. The result: root@OpenWrt:/# free total used free sharedbuffers Mem: 16628216 14864 16613352 0236 -/+ buffers:14628 16613588 Swap:0 0 0 total time spent to get to userland: about 30 min. too easy... Let's give it a try with a system with 32GB of RAM: root@OpenWrt:/# free total used free sharedbuffers Mem: 33274356 17124 33257232 0252 -/+ buffers:16872 33257484 Swap:0 0 0 d'oh! ok, I'm not entirely honest, tmpfs chocked: tmpfs: Bad value '1.70365e+10' for mount option 'size'. Let's give it a try with 60GB now: root@OpenWrt:/# free total used free sharedbuffers Mem: 62405104 17116 62387988 0256 -/+ buffers:16860 62388244 Swap:0 0 0 damn... Linux' no fun... :-( - Arnaud ps: the kernel actually panic with the full 64GB of RAM, while mounting the filesystem. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi, On Mon, Nov 14, 2011 at 5:14 AM, Andriy Gapon a...@freebsd.org wrote: on 14/11/2011 02:38 Arnaud Lacombe said the following: you (committers) I wonder how it would work out if you were made a committer and couldn't say you (committers) any more... :-) The real question is rather whether or not I would accept such a role, or better, would I ever be proposed such a role ? I doubt someone praising the Bazaar, openly challenging core and historical members, would fit in the Cathedral, even if my work is only ever gonna be in the `projects/' subdirectory. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi, On Wed, Nov 9, 2011 at 12:39 PM, Julian Elischer jul...@freebsd.org wrote: On 11/8/11 9:29 PM, Arnaud Lacombe wrote: [...] However, if you want to know, my heart tends to be with BSDs. Unfortunately, it's a sad love-story where your Beloved keeps deceiving you day after day. You want to change small bits at a time, make several iteration of progress to make things brighter, but your Beloved refuses any change because of too much inertia. Sad. mostly it's because you keep attacking your loved one with a steak knife. flowers might get you further. Well, it would would seem that keeping sending patch is what you consider attacking your loved one with a steak knife, because yes, this is what I will keep doing. so you are used to doing it that way.. but don't expect us to change just because that's what Linux does. again, mentioning Linux is totally irrelevant. Use of Instruction Pointer are implementation details for a not so intrusive solution to the problem I pointed out, and which you are totally missing. since these modules were written many new options have come. Maybe this is the real problem, FreeBSD is unable to keep up and to make interfaces written +10 years ago evolves. Worst, you (committers) keep on taking decision based on changes made 10 to 12 years ago that are totally irrelevant today, making these decisions nothing but plain bad. well, you're lucky FreeBSD supports your device! Lately, we got lately a shiny multi-queue network cards with bypass mechanism... that is not supported in FreeBSD. So currently, we got an expensive paper-weight. well write a driver for it.. my time is limited, and you (FreeBSD folks) seem to love making it even busier by your inability to make some parts of the system evolve, or by taking bad decision. This generally happens when I try to optimize some of our internal code path, hit a system bottleneck, try to prove the system is wrong, and then eventually, think about optimizing it, implement the solution one or twice, and get slammed hard when I go public with both the proof of performance hit/non-correctness/incompleteness and the correction. Unfortunately, the time complexity of the whole process is 2^O(n) :( what do you think I'm doing with the driver I'm talking about? I wrote several bypass network card drivers when I was at cisco/ironport.. it's not rocket science, though it would be nice if we were to come up with a standard interface for bypass interfaces. indeed. That is a different topic though.. indeed. 1/ half the time freebsd will just immediatly assert on something and present you with the bug.. done. well, certainly not from a release build; assertion are disabled. During development. we NEVER have bugs in production ;-) [sic.] [0]: I am able to crash any kernel between 7-STABLE to 9.STABLE within minutes, with the right pattern and (mainstream and well supported) hardware. and who have you reported that to? bug number? http://lists.freebsd.org/pipermail/freebsd-net/2011-September/029722.html I suspect PR/155597 and a few other are related. [3]: FreeBSD (8-STABLE) is way to limited and un-integrated to be anywhere but useful, not to speak about kernel bug which leave the system so fracked up that you have no other choice but to hard-reboot. bug number? usb/156725, amd64/156726 - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi,
On Mon, Nov 7, 2011 at 2:03 PM, Arnaud Lacombe lacom...@gmail.com wrote:
On Mon, Nov 7, 2011 at 4:36 AM, Attilio Rao atti...@freebsd.org wrote:
2011/11/7 Arnaud Lacombe lacom...@gmail.com:
Hi,
On Sat, Nov 5, 2011 at 10:13 AM, Kostik Belousov kostik...@gmail.com
wrote:
On Fri, Nov 04, 2011 at 06:03:39PM +0200, Kostik Belousov wrote:
Below is the KBI patch after vm_page_bits_t merge is done.
Again, I did not spent time converting all in-tree consumers
from the (potentially) loadable modules to the new KPI until it
is agreed upon.
diff --git a/sys/nfsclient/nfs_bio.c b/sys/nfsclient/nfs_bio.c
index 305c189..7264cd1 100644
--- a/sys/nfsclient/nfs_bio.c
+++ b/sys/nfsclient/nfs_bio.c
@@ -128,7 +128,7 @@ nfs_getpages(struct vop_getpages_args *ap)
* can only occur at the file EOF.
*/
VM_OBJECT_LOCK(object);
- if (pages[ap-a_reqpage]-valid != 0) {
+ if (vm_page_read_valid(pages[ap-a_reqpage]) != 0) {
for (i = 0; i npages; ++i) {
if (i != ap-a_reqpage) {
vm_page_lock(pages[i]);
@@ -198,16 +198,16 @@ nfs_getpages(struct vop_getpages_args *ap)
/*
* Read operation filled an entire page
*/
- m-valid = VM_PAGE_BITS_ALL;
- KASSERT(m-dirty == 0,
+ vm_page_write_valid(m, VM_PAGE_BITS_ALL);
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else if (size toff) {
/*
* Read operation filled a partial page.
*/
- m-valid = 0;
+ vm_page_write_valid(m, 0);
vm_page_set_valid(m, 0, size - toff);
- KASSERT(m-dirty == 0,
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else {
/*
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index 389aea5..2f41e70 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -2677,6 +2677,66 @@ vm_page_test_dirty(vm_page_t m)
vm_page_dirty(m);
}
+void
+vm_page_lock_func(vm_page_t m, const char *file, int line)
+{
+
+#if LOCK_DEBUG 0 || defined(MUTEX_NOINLINE)
+ _mtx_lock_flags(vm_page_lockptr(m), 0, file, line);
+#else
+ __mtx_lock(vm_page_lockptr(m), 0, file, line);
+#endif
+}
+
Why do you re-implement the wheel ? all the point of these assessors
is to hide implementation detail. IMO, you should restrict yourself to
the documented API from mutex(9), only.
Oh, wait, you end-up using LOCK_FILE instead of just __FILE__, but
wait LOCK_FILE is either just __FILE__, or NULL, depending on
LOCK_DEBUG, but you wouldn't have those function without
INVARIANTS This whole LOCK_FILE/LOCK_LINE seem completely
fracked-up... If you don't want this code in INVARIANTS, but in
LOCK_DEBUG, only make it live only in the LOCK_DEBUG case.
Btw, let me also question the use of non-inline functions.
My impression is that you don't really understand the patch, thus your
disrespectful words used here are really unjustified.
Well, unfortunately, I wasn't around to comment 10 years ago when this got in.
I think that kib@ intention is just to get the most official way to
pass down file and line to locking functions from the consumers.
His patch is technically right, but I would prefer something
different (see below).
Yes, and that's not an excuse to use the _internal_ implementation
details of mutex(9), and not the exposed API. This is especially true
when applied to someone so eager to follow standards.
LOCK_FILE and LOCK_LINE exist for reducing the space in .rodata
section. Without INVARIANTS/WITNESS/etc. they will just be NULL and
not pointing to a lot of datas that won't be used in the opposite
case.
You comment just as if __FILE__ and __LINE__ were mandatory in a debug
interface. This is _not_ true. __FILE__ and __LINE__ are just hideous
for debugging of anything but early alpha code. LOCK_FILE and
LOCK_LINE are a bad answer to a bad interface. Even if you just pass
NULL and 0 as argument, you've got the bloat of passing unused
argument. You might as well just pass a single argument that would do
the exact same job and be _always_ available, eg. the IP of the
caller, or the first return address. KDB magic still let you translate
to something humanly understandable. If the toolchain does not support
the feature on all supported platform, well, fix the toolchain.
To avoid future complaints about the fact that I would be only talk
without action, I did implement what I suggested above. As it is
quite a large patch-set, I will not post it directly here, however, it
is available on github
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi, On Tue, Nov 8, 2011 at 7:08 PM, Julian Elischer jul...@freebsd.org wrote: On 11/8/11 10:49 AM, Arnaud Lacombe wrote: Hi, To avoid future complaints about the fact that I would be only talk without action, I did implement what I suggested above. As it is quite a large patch-set, I will not post it directly here, however, it is available on github: https://github.com/lacombar/freebsd/tree/master/topic/kern-lock-debug It convert a bunch of debug interface to use the caller instruction pointer, as well as a proof-of-concept teaching printf(9) to convert IP to symbol_name+offset. It translates in a direct saving of about +250kB on i386's GENERIC, just in kernel text size. Even the worst case, ie LOCK_DEBUG == 0, translates to a save of +80kB. Please note that this is still WIP code. A couple of comments. Firstly, the idea of a printf method to print the IP as symbol+offset is an interesting idea that should be followed up in its own right. However, (comment 2) I would much rather file+line in this case. I don't want to have the tools to decode the offset into a location in sources. this already exists and is called debug symbols - Arnaud We have both systems in operation art work and I far prefer the latter. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi, On Tue, Nov 8, 2011 at 3:55 PM, Andriy Gapon a...@freebsd.org wrote: [cc list trimmed] on 08/11/2011 22:34 Attilio Rao said the following: 2011/11/8 Arnaud Lacombe lacom...@gmail.com: To avoid future complaints about the fact that I would be only talk without action, I did implement what I suggested above. As it is quite a large patch-set, I will not post it directly here, however, it is available on github: I really think that this is way too dependent by the good health of your tool, thus that is highly fragile. However, you may have more luck by just the core of your kernel changes here, for comment and leave alone all the (ptr - LOCK_FILE/LOCK_LINE conversion). Said that, I think this logic is too fragile and likely won't be as effective as __FILE__/__LINE__ in many cases. I agree. If we were able to somehow automatically, magically, easily and correctly determine an instruction pointer of a caller, then it would make sense to ditch explicit passing of __FILE__/__LINE__ arguments in favor of doing instruction pointer decoding. again, no need for magic, this already exists, as the form of gcc[0]'s __builtin_return_address(0). But if we are just replacing explicit passing of (well-known) macros __FILE__/__LINE__ with explicit passing of THIS_IP, then I don't see a point. make sense too, but you need to be sure stuff between the caller and callee is fully inlined. Apologies if I missed the rationale for this change. mostly getting rid of all the __FILE__ and __LINE__ bloat. - Arnaud [0]: check about LLVM support is left to the reader. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi,
On Tue, Nov 8, 2011 at 3:34 PM, Attilio Rao atti...@freebsd.org wrote:
2011/11/8 Arnaud Lacombe lacom...@gmail.com:
Hi,
On Mon, Nov 7, 2011 at 2:03 PM, Arnaud Lacombe lacom...@gmail.com wrote:
On Mon, Nov 7, 2011 at 4:36 AM, Attilio Rao atti...@freebsd.org wrote:
2011/11/7 Arnaud Lacombe lacom...@gmail.com:
Hi,
On Sat, Nov 5, 2011 at 10:13 AM, Kostik Belousov kostik...@gmail.com
wrote:
On Fri, Nov 04, 2011 at 06:03:39PM +0200, Kostik Belousov wrote:
Below is the KBI patch after vm_page_bits_t merge is done.
Again, I did not spent time converting all in-tree consumers
from the (potentially) loadable modules to the new KPI until it
is agreed upon.
diff --git a/sys/nfsclient/nfs_bio.c b/sys/nfsclient/nfs_bio.c
index 305c189..7264cd1 100644
--- a/sys/nfsclient/nfs_bio.c
+++ b/sys/nfsclient/nfs_bio.c
@@ -128,7 +128,7 @@ nfs_getpages(struct vop_getpages_args *ap)
* can only occur at the file EOF.
*/
VM_OBJECT_LOCK(object);
- if (pages[ap-a_reqpage]-valid != 0) {
+ if (vm_page_read_valid(pages[ap-a_reqpage]) != 0) {
for (i = 0; i npages; ++i) {
if (i != ap-a_reqpage) {
vm_page_lock(pages[i]);
@@ -198,16 +198,16 @@ nfs_getpages(struct vop_getpages_args *ap)
/*
* Read operation filled an entire page
*/
- m-valid = VM_PAGE_BITS_ALL;
- KASSERT(m-dirty == 0,
+ vm_page_write_valid(m, VM_PAGE_BITS_ALL);
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else if (size toff) {
/*
* Read operation filled a partial page.
*/
- m-valid = 0;
+ vm_page_write_valid(m, 0);
vm_page_set_valid(m, 0, size - toff);
- KASSERT(m-dirty == 0,
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else {
/*
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index 389aea5..2f41e70 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -2677,6 +2677,66 @@ vm_page_test_dirty(vm_page_t m)
vm_page_dirty(m);
}
+void
+vm_page_lock_func(vm_page_t m, const char *file, int line)
+{
+
+#if LOCK_DEBUG 0 || defined(MUTEX_NOINLINE)
+ _mtx_lock_flags(vm_page_lockptr(m), 0, file, line);
+#else
+ __mtx_lock(vm_page_lockptr(m), 0, file, line);
+#endif
+}
+
Why do you re-implement the wheel ? all the point of these assessors
is to hide implementation detail. IMO, you should restrict yourself to
the documented API from mutex(9), only.
Oh, wait, you end-up using LOCK_FILE instead of just __FILE__, but
wait LOCK_FILE is either just __FILE__, or NULL, depending on
LOCK_DEBUG, but you wouldn't have those function without
INVARIANTS This whole LOCK_FILE/LOCK_LINE seem completely
fracked-up... If you don't want this code in INVARIANTS, but in
LOCK_DEBUG, only make it live only in the LOCK_DEBUG case.
Btw, let me also question the use of non-inline functions.
My impression is that you don't really understand the patch, thus your
disrespectful words used here are really unjustified.
Well, unfortunately, I wasn't around to comment 10 years ago when this got
in.
I think that kib@ intention is just to get the most official way to
pass down file and line to locking functions from the consumers.
His patch is technically right, but I would prefer something
different (see below).
Yes, and that's not an excuse to use the _internal_ implementation
details of mutex(9), and not the exposed API. This is especially true
when applied to someone so eager to follow standards.
LOCK_FILE and LOCK_LINE exist for reducing the space in .rodata
section. Without INVARIANTS/WITNESS/etc. they will just be NULL and
not pointing to a lot of datas that won't be used in the opposite
case.
You comment just as if __FILE__ and __LINE__ were mandatory in a debug
interface. This is _not_ true. __FILE__ and __LINE__ are just hideous
for debugging of anything but early alpha code. LOCK_FILE and
LOCK_LINE are a bad answer to a bad interface. Even if you just pass
NULL and 0 as argument, you've got the bloat of passing unused
argument. You might as well just pass a single argument that would do
the exact same job and be _always_ available, eg. the IP of the
caller, or the first return address. KDB magic still let you translate
to something humanly understandable. If the toolchain does not support
the feature on all supported platform, well, fix the toolchain.
To avoid future complaints about the fact that I would be only talk
without action, I did implement
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi, On Tue, Nov 8, 2011 at 8:09 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Tue, Nov 8, 2011 at 3:55 PM, Andriy Gapon a...@freebsd.org wrote: [cc list trimmed] on 08/11/2011 22:34 Attilio Rao said the following: 2011/11/8 Arnaud Lacombe lacom...@gmail.com: To avoid future complaints about the fact that I would be only talk without action, I did implement what I suggested above. As it is quite a large patch-set, I will not post it directly here, however, it is available on github: I really think that this is way too dependent by the good health of your tool, thus that is highly fragile. However, you may have more luck by just the core of your kernel changes here, for comment and leave alone all the (ptr - LOCK_FILE/LOCK_LINE conversion). Said that, I think this logic is too fragile and likely won't be as effective as __FILE__/__LINE__ in many cases. I agree. If we were able to somehow automatically, magically, easily and correctly determine an instruction pointer of a caller, then it would make sense to ditch explicit passing of __FILE__/__LINE__ arguments in favor of doing instruction pointer decoding. again, no need for magic, this already exists, as the form of gcc[0]'s __builtin_return_address(0). actually, this should be __builtin_return_address(1). - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi, On Tue, Nov 8, 2011 at 7:08 PM, Julian Elischer jul...@freebsd.org wrote: On 11/8/11 10:49 AM, Arnaud Lacombe wrote: Hi, To avoid future complaints about the fact that I would be only talk without action, I did implement what I suggested above. As it is quite a large patch-set, I will not post it directly here, however, it is available on github: https://github.com/lacombar/freebsd/tree/master/topic/kern-lock-debug It convert a bunch of debug interface to use the caller instruction pointer, as well as a proof-of-concept teaching printf(9) to convert IP to symbol_name+offset. It translates in a direct saving of about +250kB on i386's GENERIC, just in kernel text size. Even the worst case, ie LOCK_DEBUG == 0, translates to a save of +80kB. Please note that this is still WIP code. A couple of comments. Firstly, the idea of a printf method to print the IP as symbol+offset is an interesting idea that should be followed up in its own right. FWIW, I have no credit in this idea. It has been in Linux for ages and ages. That said, IP address are barely used in FreeBSD, there is no legacy. As such, the API should not use `unsigned long' but `void *'[0]; this is the natural type returned by `__builtin_return_address()' and the `' operator. This would allow to introduce a modifier to `%p' to do the translation. - Arnaud ps: netgraph is on my target list, as well as the list code, to some extend :) [0]: as I really hate `caddr_t' ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Using Instruction Pointer address in debug interfaces [Was: Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]]
Hi, On Tue, Nov 8, 2011 at 9:35 PM, Julian Elischer jul...@freebsd.org wrote: On 11/8/11 5:52 PM, Arnaud Lacombe wrote: Hi, On Tue, Nov 8, 2011 at 7:08 PM, Julian Elischerjul...@freebsd.org wrote: On 11/8/11 10:49 AM, Arnaud Lacombe wrote: Hi, To avoid future complaints about the fact that I would be only talk without action, I did implement what I suggested above. As it is quite a large patch-set, I will not post it directly here, however, it is available on github: https://github.com/lacombar/freebsd/tree/master/topic/kern-lock-debug It convert a bunch of debug interface to use the caller instruction pointer, as well as a proof-of-concept teaching printf(9) to convert IP to symbol_name+offset. It translates in a direct saving of about +250kB on i386's GENERIC, just in kernel text size. Even the worst case, ie LOCK_DEBUG == 0, translates to a save of +80kB. Please note that this is still WIP code. A couple of comments. Firstly, the idea of a printf method to print the IP as symbol+offset is an interesting idea that should be followed up in its own right. FWIW, I have no credit in this idea. It has been in Linux for ages and ages. yeah as I said at work I use linux and BSD... the linux stuff that just prints out IP really annoys me. the list stuff and netgraph debug (which should be off in any production system) this is, I guess, where we do not agree. You find it acceptable to run totally different code in production and during debug. I do not. This is completely insane, even more nowadays where heavy parallelism increases the likelihood of races, and subtle change in the code, even optimization, can cause total behavioral change (ie. Heisenbug). For the record, we have been tracking for more than 2 months (first occurrences happened a year ago) an mbuf corruption in the network stack, present in all released code since at least FreeBSD 7[0]. Each time we think it is fixed, we are proven wrong by customers within a few days when the system crashes again. Even the last attempt which was believed to be bullet-proof failed and crashes daily. All that to say that production code should embed enough facilities to allow the system to be fully debugged with a runtime cost as low as possible, and a code-size cost as low as possible[1]. I should be able to connect on a production machine, turn a knob, an see what is going wrong, without the customer noticing. In the worst case, when you have to enable debug-only code, it must not be done by making the non-debug case more expensive, but wrap around. The whole original point of the patches was that LOCK_FILE and LOCK_LINE are a bad answer to a wrong problem. `__FILE__, __LINE__' and the bloat introduced is not the problem, `const char *file, int line' in way too much prototypes is. Now, you make me realize that `const char *file, int line' should just be removed, not replaced by `unsigned long' or anything else. It's likely to be done in another iteration. just require you to be able to see the console. and have sources nearby. if you need the IP use gdb. console debugging is yet another abomination which should be hunted down. Just try to do any useful work at high-pps on a serial console... it's just what you are used to. You are obviously from the dark side ^H^H^H^H^H^H linux. My obedience is totally irrelevant to the problem. However, if you want to know, my heart tends to be with BSDs. Unfortunately, it's a sad love-story where your Beloved keeps deceiving you day after day. You want to change small bits at a time, make several iteration of progress to make things brighter, but your Beloved refuses any change because of too much inertia. Sad. so you are used to doing it that way.. but don't expect us to change just because that's what Linux does. again, mentioning Linux is totally irrelevant. Use of Instruction Pointer are implementation details for a not so intrusive solution to the problem I pointed out, and which you are totally missing. Now, please answer this: do you find any of the bloat to the non-debug case (ie. passing a NULL pointer and a 0 integer, when `LOCK_DEBUG == 0') worth the extra debugability comfort to be acceptable ? If you do, then your focus is on making things comfortable for developers, at the expense 100's of users, rather than making things comfortable for 100's of users, at the expense of developers. When we have a problem at work on teh Linux driver, my first step is always to try duplicate it on FreeBSD because: well, you're lucky FreeBSD supports your device! Lately, we got lately a shiny multi-queue network cards with bypass mechanism... that is not supported in FreeBSD. So currently, we got an expensive paper-weight. 1/ half the time freebsd will just immediatly assert on something and present you with the bug.. done. well, certainly not from a release build; assertion are disabled. 2/ I can run gdb through firewire on it on ANY standard unmodified kernel
Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]
On Mon, Nov 7, 2011 at 4:36 AM, Attilio Rao atti...@freebsd.org wrote:
2011/11/7 Arnaud Lacombe lacom...@gmail.com:
Hi,
On Sat, Nov 5, 2011 at 10:13 AM, Kostik Belousov kostik...@gmail.com wrote:
On Fri, Nov 04, 2011 at 06:03:39PM +0200, Kostik Belousov wrote:
Below is the KBI patch after vm_page_bits_t merge is done.
Again, I did not spent time converting all in-tree consumers
from the (potentially) loadable modules to the new KPI until it
is agreed upon.
diff --git a/sys/nfsclient/nfs_bio.c b/sys/nfsclient/nfs_bio.c
index 305c189..7264cd1 100644
--- a/sys/nfsclient/nfs_bio.c
+++ b/sys/nfsclient/nfs_bio.c
@@ -128,7 +128,7 @@ nfs_getpages(struct vop_getpages_args *ap)
* can only occur at the file EOF.
*/
VM_OBJECT_LOCK(object);
- if (pages[ap-a_reqpage]-valid != 0) {
+ if (vm_page_read_valid(pages[ap-a_reqpage]) != 0) {
for (i = 0; i npages; ++i) {
if (i != ap-a_reqpage) {
vm_page_lock(pages[i]);
@@ -198,16 +198,16 @@ nfs_getpages(struct vop_getpages_args *ap)
/*
* Read operation filled an entire page
*/
- m-valid = VM_PAGE_BITS_ALL;
- KASSERT(m-dirty == 0,
+ vm_page_write_valid(m, VM_PAGE_BITS_ALL);
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else if (size toff) {
/*
* Read operation filled a partial page.
*/
- m-valid = 0;
+ vm_page_write_valid(m, 0);
vm_page_set_valid(m, 0, size - toff);
- KASSERT(m-dirty == 0,
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else {
/*
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index 389aea5..2f41e70 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -2677,6 +2677,66 @@ vm_page_test_dirty(vm_page_t m)
vm_page_dirty(m);
}
+void
+vm_page_lock_func(vm_page_t m, const char *file, int line)
+{
+
+#if LOCK_DEBUG 0 || defined(MUTEX_NOINLINE)
+ _mtx_lock_flags(vm_page_lockptr(m), 0, file, line);
+#else
+ __mtx_lock(vm_page_lockptr(m), 0, file, line);
+#endif
+}
+
Why do you re-implement the wheel ? all the point of these assessors
is to hide implementation detail. IMO, you should restrict yourself to
the documented API from mutex(9), only.
Oh, wait, you end-up using LOCK_FILE instead of just __FILE__, but
wait LOCK_FILE is either just __FILE__, or NULL, depending on
LOCK_DEBUG, but you wouldn't have those function without
INVARIANTS This whole LOCK_FILE/LOCK_LINE seem completely
fracked-up... If you don't want this code in INVARIANTS, but in
LOCK_DEBUG, only make it live only in the LOCK_DEBUG case.
Btw, let me also question the use of non-inline functions.
My impression is that you don't really understand the patch, thus your
disrespectful words used here are really unjustified.
Well, unfortunately, I wasn't around to comment 10 years ago when this got in.
I think that kib@ intention is just to get the most official way to
pass down file and line to locking functions from the consumers.
His patch is technically right, but I would prefer something
different (see below).
Yes, and that's not an excuse to use the _internal_ implementation
details of mutex(9), and not the exposed API. This is especially true
when applied to someone so eager to follow standards.
LOCK_FILE and LOCK_LINE exist for reducing the space in .rodata
section. Without INVARIANTS/WITNESS/etc. they will just be NULL and
not pointing to a lot of datas that won't be used in the opposite
case.
You comment just as if __FILE__ and __LINE__ were mandatory in a debug
interface. This is _not_ true. __FILE__ and __LINE__ are just hideous
for debugging of anything but early alpha code. LOCK_FILE and
LOCK_LINE are a bad answer to a bad interface. Even if you just pass
NULL and 0 as argument, you've got the bloat of passing unused
argument. You might as well just pass a single argument that would do
the exact same job and be _always_ available, eg. the IP of the
caller, or the first return address. KDB magic still let you translate
to something humanly understandable. If the toolchain does not support
the feature on all supported platform, well, fix the toolchain.
I'm unsure if this replies to your concerns because you just criticize
without making a real technical question in this post.
I made comments on 3 points:
- using internal implementation details of mutex(9) is broken
- LOCK_FILE/LOCK_LINE are broken (a bit of a divergence on the
original subject :/)
- there is _no_
Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3
Hi,
On Wed, Nov 2, 2011 at 6:32 AM, Andriy Gapon a...@freebsd.org wrote:
[restored cc: to the original poster]
on 02/11/2011 08:10 Benjamin Kaduk said the following:
I am perhaps confused. Last I checked, bsd.kmod.mk caused '-include
opt_global.h' to be passed on the command line. Is the issue just that the
opt_global.h used for the kmod could be different from the actual kernel's
opt_global.h, because KERNCONF was not specified and the header is generated
at
module-build time? In this case, clearly the onus is on the user to pass
KERNCONF at module build time.
To be precise, this is what is actually passed to a compiler:
sys/conf/kmod.mk:
.if defined(KERNBUILDDIR)
CFLAGS+= -DHAVE_KERNEL_OPTION_HEADERS -include
${KERNBUILDDIR}/opt_global.h
.endif
where KERNBUILDDIR can be passed via environment from a kernel build:
sys/conf/kern.post.mk:
MKMODULESENV+= KERNBUILDDIR=${.CURDIR} SYSDIR=${SYSDIR}
KERNCONF does not have any meaning in a module build.
To make sure that a module build sees exactly the same kernel options as a
kernel with which the module should work, one has to either build the module
together with the kernel (within the kernel build; search for MODULES in
make.conf(5)) or to manually specify KERNBUILDDIR to point to a correct kernel
build directory. (Which to a certain degree implies impossibility to build a
perfect module for a pre-built binary kernel or to provide a perfect
universal pre-built module for any custom kernel)
Of course, the real problem is that modules should not care about any (or at
least some) kernel options, they should be isolated from the options via a
proper KPI/KBI (perhaps DDI or module-to-kernel interface or whatever). A
module should be able to work correctly with kernels built with different
options.
You cannot be make a point in shade of gray, it either must care or
must not care about kernel option, not care about some, but not
other. Moreover, you cannot advocate stable internal KBI/KPI when you
are not even able to provide a stable userland ABI...
P.P.S. [and tangential] I see that many module makefiles fake up various
kernel
options in a fashion similar to the following:
.if !defined(KERNBUILDDIR)
opt_compat.h:
echo #define COMPAT_FREEBSD6 1 ${.TARGET}
opt_kbd.h:
echo #define KBD_INSTALL_CDEV 1 ${.TARGET}
.endif
And handful of modules fake up opt_global.h, e.g.:
opt_global.h:
echo #define ALTQ 1 ${.TARGET}
personal opinionThis mess is utterly broken./personal opinion
FWIW, I advocate to make KERNBUILDDIR (ie. kernel option's
configuration) mandatory for building any modules.
- Arnaud
--
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]
Hi, On Mon, Nov 7, 2011 at 2:03 PM, Arnaud Lacombe lacom...@gmail.com wrote: On Mon, Nov 7, 2011 at 4:36 AM, Attilio Rao atti...@freebsd.org wrote: I'm unsure if this replies to your concerns because you just criticize without making a real technical question in this post. I made comments on 3 points: - using internal implementation details of mutex(9) is broken - LOCK_FILE/LOCK_LINE are broken (a bit of a divergence on the original subject :/) - there is _no_ reason not to use inlines function for such trivial oneliners ok, I read the original thread, now that I understand the purpose of the patch. It would make the third comment irrelevant, but I still do not agree about the reason of the patch. - ARnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]
Hi,
On Mon, Nov 7, 2011 at 2:35 PM, Kostik Belousov kostik...@gmail.com wrote:
On Mon, Nov 07, 2011 at 11:45:38AM -0600, Alan Cox wrote:
Ok. I'll offer one final suggestion. Please consider an alternative
suffix to func. Perhaps, kbi or KBI. In other words, something
that hints at the function's reason for existing.
Sure. Below is the extraction of only vm_page_lock() bits, together
with the suggested rename. When Attilio provides the promised simplification
of the mutex KPI, this can be reduced.
If you want to be pedantic, you also must hide the definition of
`struct vm_page' when KLD_MODULE is defined. You want to be sure no
one is gonna mess with the internal of the structure (ie. either
directly dereference fields, compute size or any offset...) and will
have no other choice but to use assessors.
Maybe you are addressing this in another patch.
- Arnaud
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index 389aea5..ea4ea34 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -2677,6 +2677,44 @@ vm_page_test_dirty(vm_page_t m)
vm_page_dirty(m);
}
+void
+vm_page_lock_KBI(vm_page_t m, const char *file, int line)
+{
+
+#if LOCK_DEBUG 0 || defined(MUTEX_NOINLINE)
+ _mtx_lock_flags(vm_page_lockptr(m), 0, file, line);
+#else
+ __mtx_lock(vm_page_lockptr(m), 0, file, line);
+#endif
+}
+
+void
+vm_page_unlock_KBI(vm_page_t m, const char *file, int line)
+{
+
+#if LOCK_DEBUG 0 || defined(MUTEX_NOINLINE)
+ _mtx_unlock_flags(vm_page_lockptr(m), 0, file, line);
+#else
+ __mtx_unlock(vm_page_lockptr(m), curthread, 0, file, line);
+#endif
+}
+
+int
+vm_page_trylock_KBI(vm_page_t m, const char *file, int line)
+{
+
+ return (_mtx_trylock(vm_page_lockptr(m), 0, file, line));
+}
+
+void
+vm_page_lock_assert_KBI(vm_page_t m, int a, const char *file, int line)
+{
+
+#ifdef INVARIANTS
+ _mtx_assert(vm_page_lockptr(m), a, file, line);
+#endif
+}
+
int so_zerocp_fullpage = 0;
/*
diff --git a/sys/vm/vm_page.h b/sys/vm/vm_page.h
index 7099b70..a5604b7 100644
--- a/sys/vm/vm_page.h
+++ b/sys/vm/vm_page.h
@@ -218,11 +218,23 @@ extern struct vpglocks pa_lock[];
#define PA_LOCK_ASSERT(pa, a) mtx_assert(PA_LOCKPTR(pa), (a))
+#ifdef KLD_MODULE
+#define vm_page_lock(m) vm_page_lock_KBI((m), LOCK_FILE,
LOCK_LINE)
+#define vm_page_unlock(m) vm_page_unlock_KBI((m), LOCK_FILE,
LOCK_LINE)
+#define vm_page_trylock(m) vm_page_trylock_KBI((m), LOCK_FILE,
LOCK_LINE)
+#ifdef INVARIANTS
+#define vm_page_lock_assert(m, a) \
+ vm_page_lock_assert_KBI((m), (a), LOCK_FILE, LOCK_LINE)
+#else
+#define vm_page_lock_assert(m, a)
+#endif
+#else /* KLD_MODULE */
#define vm_page_lockptr(m) (PA_LOCKPTR(VM_PAGE_TO_PHYS((m
#define vm_page_lock(m) mtx_lock(vm_page_lockptr((m)))
#define vm_page_unlock(m) mtx_unlock(vm_page_lockptr((m)))
#define vm_page_trylock(m) mtx_trylock(vm_page_lockptr((m)))
#define vm_page_lock_assert(m, a)
mtx_assert(vm_page_lockptr((m)), (a))
+#endif
#define vm_page_queue_free_mtx vm_page_queue_free_lock.data
/*
@@ -403,6 +415,11 @@ void vm_page_cowfault (vm_page_t);
int vm_page_cowsetup(vm_page_t);
void vm_page_cowclear (vm_page_t);
+void vm_page_lock_KBI(vm_page_t m, const char *file, int line);
+void vm_page_unlock_KBI(vm_page_t m, const char *file, int line);
+int vm_page_trylock_KBI(vm_page_t m, const char *file, int line);
+void vm_page_lock_assert_KBI(vm_page_t m, int a, const char *file, int line);
+
#ifdef INVARIANTS
void vm_page_object_lock_assert(vm_page_t m);
#define VM_PAGE_OBJECT_LOCK_ASSERT(m) vm_page_object_lock_assert(m)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]
Hi,
On Sat, Nov 5, 2011 at 10:13 AM, Kostik Belousov kostik...@gmail.com wrote:
On Fri, Nov 04, 2011 at 06:03:39PM +0200, Kostik Belousov wrote:
Below is the KBI patch after vm_page_bits_t merge is done.
Again, I did not spent time converting all in-tree consumers
from the (potentially) loadable modules to the new KPI until it
is agreed upon.
diff --git a/sys/nfsclient/nfs_bio.c b/sys/nfsclient/nfs_bio.c
index 305c189..7264cd1 100644
--- a/sys/nfsclient/nfs_bio.c
+++ b/sys/nfsclient/nfs_bio.c
@@ -128,7 +128,7 @@ nfs_getpages(struct vop_getpages_args *ap)
* can only occur at the file EOF.
*/
VM_OBJECT_LOCK(object);
- if (pages[ap-a_reqpage]-valid != 0) {
+ if (vm_page_read_valid(pages[ap-a_reqpage]) != 0) {
for (i = 0; i npages; ++i) {
if (i != ap-a_reqpage) {
vm_page_lock(pages[i]);
@@ -198,16 +198,16 @@ nfs_getpages(struct vop_getpages_args *ap)
/*
* Read operation filled an entire page
*/
- m-valid = VM_PAGE_BITS_ALL;
- KASSERT(m-dirty == 0,
+ vm_page_write_valid(m, VM_PAGE_BITS_ALL);
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else if (size toff) {
/*
* Read operation filled a partial page.
*/
- m-valid = 0;
+ vm_page_write_valid(m, 0);
vm_page_set_valid(m, 0, size - toff);
- KASSERT(m-dirty == 0,
+ KASSERT(vm_page_read_dirty(m) == 0,
(nfs_getpages: page %p is dirty, m));
} else {
/*
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index 389aea5..2f41e70 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -2677,6 +2677,66 @@ vm_page_test_dirty(vm_page_t m)
vm_page_dirty(m);
}
+void
+vm_page_lock_func(vm_page_t m, const char *file, int line)
+{
+
+#if LOCK_DEBUG 0 || defined(MUTEX_NOINLINE)
+ _mtx_lock_flags(vm_page_lockptr(m), 0, file, line);
+#else
+ __mtx_lock(vm_page_lockptr(m), 0, file, line);
+#endif
+}
+
Why do you re-implement the wheel ? all the point of these assessors
is to hide implementation detail. IMO, you should restrict yourself to
the documented API from mutex(9), only.
Oh, wait, you end-up using LOCK_FILE instead of just __FILE__, but
wait LOCK_FILE is either just __FILE__, or NULL, depending on
LOCK_DEBUG, but you wouldn't have those function without
INVARIANTS This whole LOCK_FILE/LOCK_LINE seem completely
fracked-up... If you don't want this code in INVARIANTS, but in
LOCK_DEBUG, only make it live only in the LOCK_DEBUG case.
Btw, let me also question the use of non-inline functions.
+void
+vm_page_unlock_func(vm_page_t m, const char *file, int line)
+{
+
+#if LOCK_DEBUG 0 || defined(MUTEX_NOINLINE)
+ _mtx_unlock_flags(vm_page_lockptr(m), 0, file, line);
+#else
+ __mtx_unlock(vm_page_lockptr(m), curthread, 0, file, line);
+#endif
+}
+
+int
+vm_page_trylock_func(vm_page_t m, const char *file, int line)
+{
+
+ return (_mtx_trylock(vm_page_lockptr(m), 0, file, line));
+}
+
+void
+vm_page_lock_assert_func(vm_page_t m, int a, const char *file, int line)
+{
+
+#ifdef INVARIANTS
+ _mtx_assert(vm_page_lockptr(m), a, file, line);
+#endif
+}
+
same remark on all the above.
+vm_page_bits_t
+vm_page_read_dirty_func(vm_page_t m)
+{
+
+ return (m-dirty);
+}
+
+vm_page_bits_t
+vm_page_read_valid_func(vm_page_t m)
+{
+
+ return (m-valid);
+}
+
+void
+vm_page_write_valid_func(vm_page_t m, vm_page_bits_t v)
+{
+
+ m-valid = v;
+}
+
+
int so_zerocp_fullpage = 0;
/*
diff --git a/sys/vm/vm_page.h b/sys/vm/vm_page.h
index 7099b70..4f8f71e 100644
--- a/sys/vm/vm_page.h
+++ b/sys/vm/vm_page.h
@@ -218,12 +218,50 @@ extern struct vpglocks pa_lock[];
#define PA_LOCK_ASSERT(pa, a) mtx_assert(PA_LOCKPTR(pa), (a))
+#ifdef KLD_MODULE
+#define vm_page_lock(m) vm_page_lock_func((m), LOCK_FILE,
LOCK_LINE)
+#define vm_page_unlock(m) vm_page_unlock_func((m), LOCK_FILE,
LOCK_LINE)
+#define vm_page_trylock(m) vm_page_trylock_func((m), LOCK_FILE,
LOCK_LINE)
+#ifdef INVARIANTS
+#define vm_page_lock_assert(m, a) \
+ vm_page_lock_assert_func((m), (a), LOCK_FILE, LOCK_LINE)
+#else
+#define vm_page_lock_assert(m, a)
+#endif
+
+#define vm_page_read_dirty(m) vm_page_read_dirty_func((m))
+#define vm_page_read_valid(m) vm_page_read_valid_func((m))
+#define vm_page_write_valid(m, v) vm_page_write_valid_func((m),
(v))
+
Re: vm_page_t related KBI [Was: Re: panic at vm_page_wire with FreeBSD 9.0 Beta 3]
Hi, On Sun, Nov 6, 2011 at 11:42 AM, Kostik Belousov kostik...@gmail.com wrote: On Sun, Nov 06, 2011 at 07:22:51AM -0800, m...@freebsd.org wrote: On Sun, Nov 6, 2011 at 4:43 AM, Kostik Belousov kostik...@gmail.com wrote: Regarding the _vm_page_lock() vs. vm_page_lock_func(), the mutex.h has a lot of violations in regard of the namespaces, IMO. The __* namespace is reserved for the language implementation, so our freestanding program (kernel) ignores the requirements of the C standard with the names like __mtx_lock_spin(). Using the name _vm_page_lock() is valid, but makes it not unreasonable for other developers to introduce reserved names. So I decided to use the suffixes. vm_map.h locking is free of these violations. I'm pretty sure that when the C standard says, the implementation, they're referring to the compiler and OS it runs on. Which makes the FreeBSD kernel part of the implementation, which is precisely why so many headers have defines that start with __ and then, if certain posix defines are set, also uses non-__ versions of the name. For libc providing parts, required by standard, you are right. But our kernel is a freestanding program using a compiler, so in-kernel uses of the reserved namespace is a violation. So you prefer to introduce a new notation which will confuses everybody for the sake of following an interpretation of the standard[0] ? Btw, which point of the standard are you quoting ? Section 7.1.3 Reserved identifiers of ISO/IEC 9899 ? Thanks, - Arnaud ps: my vote is for a deep-sky-blue bikeshed. [0]: I'd be tempted to interpret the implementation as the non-visible part of an API, ie vm_page_lock() is public and rely on __vm_page_lock() for its implementation. As such, I would not consider the kernel as a single whole unit, but as a sum of public API and implementation. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: request: merging if_ath_tx branch to HEAD
Hi, On Thu, Nov 3, 2011 at 3:44 AM, Adrian Chadd adr...@freebsd.org wrote: On 31 October 2011 20:15, Doug Barton do...@freebsd.org wrote: In any case, I do want to merge the ath 11n stuff into -9, so even if it's not done by 9.0, it'll be done shortly after. Given that RC1 is already out, you should probably check with re@ first. I'll poke them tomorrow, but since others have merged in driver changes and other stuff into -HEAD, I wouldn't be the first person to merge in bigger things. Maybe not the place to ask, but why are you (ie. FreeBSD folks) unable to unleash commit to `trunk' and let only required MFC go to `stable/9' ? - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: FreeBSD 9.0 amd64 RC1 and KDE4
Hi, On Wed, Oct 26, 2011 at 7:09 PM, Mehmet Erol Sanliturk m.e.sanlit...@gmail.com wrote: The KDE4 in FreeBSD 9.0 RC1 amd64 is generating enormous amount of error messages during usage ( not visible on screen , but seen after Ctrl-Alt-F1 discontinuation of X ) . This is making it extremely slow which may be considered to be practically unusable . Actually parts are working generally but every step is waiting so much that such a usage is not practically applicable . What are the message(s) ? Thanks, - Arnaud The KDE4 in FreeBSD 9.0 RC1 i386 is working satisfactorily , but for memory requirements , it is not an alternative . There are important usability differences between both architectures with respect to KDE4 execution . One alternative is mentioned as PC-BSD , but installation of PC-BSD is NOT possible : In PC-BSD 8.2 amd64 Release and following snapshot(s) , within 18 ( eighteen ) hours , only a small percent ( less than % 10 ) could be installed . Now , in PC-BSD 9.0 RC1 amd64 , I am still waiting : In 4 ( four ) hours it could come to Installing Meta Package : base-system . Such an installation structure is really unusable . I will discontinue it because it seems that complete install will require many days with that speed . I am continuously installing many other distributions which mostly they are consuming time around thirty minutes ( except upgrade during installation , if it is selected ) . Thirty minutes may be considered an acceptable duration for installation , let's say , time less than one hour as endurable . It should not be forgotten that , the task is copy of approximately 4 Giga Bytes to hard disk from a DVD-Rom with an additional decompressing of files . Therefore , for the KDE4 users in the amd64 platform , there is a big problem . This was also the case for 8.2 amd64 Release . Thank you very much . Mehmet Erol Sanliturk ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: /usr/src/sys/conf/kern.mk, line 10: Malformed conditional (${FREEBSD_GCC}), /usr/src/sys/conf/kern.mk, line 14: if-less endif
Hi,
On Mon, Oct 24, 2011 at 6:23 PM, Hartmann, O.
ohart...@zedat.fu-berlin.de wrote:
On 10/24/11 00:38, Garrett Cooper wrote:
On Oct 23, 2011, at 3:31 PM, Hartmann, O. wrote:
Kernel building fails since today when kernel gets compiled via CLANG:
--
stage 2.1: cleaning up the object tree
--
cd /usr/obj/usr/src/sys/THOR; MAKEOBJDIRPREFIX=/usr/obj
MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE=native
GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin
GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font
GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac
_SHLIBDIRPREFIX=/usr/obj/usr/src/tmp VERSION=FreeBSD 10.0-CURRENT
amd64 100 INSTALL=sh /usr/src/tools/install.sh
PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/
usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/usr
/sbin:/usr/obj/usr/src/tmp/usr/bin:/usr/obj/usr/src/tmp/usr/games:/sbi
n:/bin:/usr/sbin:/usr/bin /usr/obj/usr/src/make.amd64/make
KERNEL=kernel cleandir
/usr/src/sys/conf/kern.mk, line 10: Malformed conditional
(${FREEBSD_GCC})
/usr/src/sys/conf/kern.mk, line 14: if-less endif
make: fatal errors encountered -- cannot continue
*** Error code 1
Stop in /usr/src.
*** Error code 1
Stop in /usr/src.
It was noted not too long ago on the commit list as well; r226665 caused the
breakage.
-Garrett___
So, this is by intention?
When does the problem disappear, so folks building FreeBSD with CLANG
are again capable of building a world and kernel?
seemed to have been fixed by dim@:
commit 2286e401073a60babb3cc8efce52657f6fa92f7e
Author: dim d...@freebsd.org
Date: Mon Oct 24 18:35:16 2011 +
Put in a temporary band-aid to fix kernel builds when CC=clang, after
r226665.
Thanks dim@!
- Arnaud
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: aliasing (or renaming) kern.geom.debugflags
Hi, 2011/10/7 Andrey V. Elsukov bu7c...@yandex.ru: On 07.10.2011 23:41, Glen Barber wrote: In my experience, without kern.geom.debugflags=16, the MBR will not be written to the memstick, leaving you with what would effectively be a coaster in the not-so-distant past. The problem is that this bad suggestion is everywhere in the Internet. And users use it always even when it not needed. I think it is bad idea add it in the our official documentation. FWIW, boot0cfg(8) also mentions it: NOTE Protection mechanisms in the geom(4) subsystem might prevent boot0cfg from being able to update the MBR on a mounted disk. Instructions for temporarily disabling these protection mechanisms can be found in the geom(4) manpage. Specifically, do a sysctl kern.geom.debugflags=0x10 to allow writing to the MBR, and restore it to 0 afterwards. - Arnaud When you doing all in the right way using debugflags=16 is not needed. And it is really dangerous when you doing something what you do not know exactly. -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: ipmi(4)/isa woes
Hi, On Fri, Oct 21, 2011 at 4:19 PM, John Baldwin j...@freebsd.org wrote: On Tuesday, October 11, 2011 6:53:11 pm Arnaud Lacombe wrote: Hi, On Tue, Oct 11, 2011 at 6:34 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi folks, I've got a machine where ipmi(4) seem to be unable to fully attach. 10-current kernel complains the following way: ipmi0: IPMI System Interface at iomem 0-0x1 on isa0 ipmi0: KCS mode found at mem 0x0 alignment 0x1 on isa ipmi0: couldn't configure I/O resource device_attach: ipmi0 attach returned 6 Resource 0 is not right and will not work. In this case your BIOS has a buggy SMBIOS / DMI table entry for IPMI. Actually, I can bypass this issue by enabling acpi(4): ipmi0: IPMI System Interface port 0xca2,0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 pmtimer0 on isa0 ipmi1: IPMI System Interface on isa0 device_attach: ipmi1 attach returned 16 You can ignore the ipmi1 messages. However, the driver fails right after with: ipmi0: Timed out waiting for GET_DEVICE_ID Are you sure you have working IPMI? The timeouts and the busted DMI table is consistent with a machine where IPMI is available via an optional daughterboard, but the daughterboard isn't installed. Well, I'm not sure exactly what's the hardware layout looks like[0], but when ipmi0 is attached on isa0, I can access the IPMI controller, get the various information, set/reset the watchdog. ipmitool(1) works like a charm. However, at some point, the KCS is unable to reach the IPMI controller. Typical error is: ipmi0: KCS: Reply address mismatch ipmi0: KCS error: 01 ipmi0: KCS: Reply address mismatch ipmi0: KCS error: 01 ipmi0: Failed to set watchdog over an undetermined period of time. Sometimes it stops before the watchdog triggers, sometimes it [falsely] triggers the watchdog. - Arnaud [0]: I'll have a look to the physical machine and let you know. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: possible mountroot regression
Hi, On Wed, Oct 19, 2011 at 3:41 PM, Garrett Cooper yaneg...@gmail.com wrote: On Wed, Oct 19, 2011 at 12:12 PM, Warren Block wbl...@wonkity.com wrote: On Wed, 19 Oct 2011, Oliver Pinter wrote: On 10/19/11, Olivier Smedts oliv...@gid0.org wrote: 2011/10/19 Marcel Moolenaar mar...@xcllnt.net: On Oct 18, 2011, at 9:04 AM, Andriy Gapon wrote: Would you be able to commit a variant of this patch sans the 'x' part? Yes, soonish. If people like the 'x' change I can do that in a followup commit as well. I just need to know if people like it or not... Yes, it's useful. But why not q for quit ? Just a bikeshed color idea... eXit :) In some languages... More important to me is the the Abort manual input which tells what it does but not why the user would want to do that. Abort manual input... and then what? Hang? Retry? Panic? Reboot? Resume attempting to mount the root device that was expected? well, panic, there isn't much other thing to do. At the very least, letting the user input something is still better than what Linux do, which is to panic. Or just go back to status quo for previous releases and we can worry about usability later? which status-quo ? the mountroot procedure of 7 is broken as well, as I found out yesterday. So what ? 6 ? 5 ? 4 ? - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: possible mountroot regression
Hi, On Fri, Oct 21, 2011 at 1:48 AM, Garrett Cooper yaneg...@gmail.com wrote: On Thu, Oct 20, 2011 at 10:44 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Wed, Oct 19, 2011 at 3:41 PM, Garrett Cooper yaneg...@gmail.com wrote: On Wed, Oct 19, 2011 at 12:12 PM, Warren Block wbl...@wonkity.com wrote: On Wed, 19 Oct 2011, Oliver Pinter wrote: On 10/19/11, Olivier Smedts oliv...@gid0.org wrote: 2011/10/19 Marcel Moolenaar mar...@xcllnt.net: On Oct 18, 2011, at 9:04 AM, Andriy Gapon wrote: Would you be able to commit a variant of this patch sans the 'x' part? Yes, soonish. If people like the 'x' change I can do that in a followup commit as well. I just need to know if people like it or not... Yes, it's useful. But why not q for quit ? Just a bikeshed color idea... eXit :) In some languages... More important to me is the the Abort manual input which tells what it does but not why the user would want to do that. Abort manual input... and then what? Hang? Retry? Panic? Reboot? Resume attempting to mount the root device that was expected? well, panic, there isn't much other thing to do. At the very least, letting the user input something is still better than what Linux do, which is to panic. Or just go back to status quo for previous releases and we can worry about usability later? which status-quo ? the mountroot procedure of 7 is broken as well, as I found out yesterday. So what ? 6 ? 5 ? 4 ? The status quo that I was thinking of was press enter to get a list of available devices instead of immediately panicking like the new bootloader code currently does. As long as that status quo is restored, I think a lot of existing users will be happy. Otherwise this discussion could stall and turn into an unnecessary bikeshed. this discussion should not even have happen, and I'd hope[0] that the next message in this thread is either a new patch, or a revision ID where the stuff is fixed, ie. less talk, more action. Btw, if no action is taken, I do not really care, I'm running my patch and am happy with it. - Arnaud [0]: which is pretty much worthless ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] Enable nxstack by default
Hi,
On Tue, Oct 18, 2011 at 5:07 AM, Kostik Belousov kostik...@gmail.com wrote:
On Mon, Oct 17, 2011 at 09:30:56PM +0200, Oliver Pinter wrote:
Hi all!
I think, it's the time to enable the nxstack feature. Any comments,
pros, cons?
I dragged the change long enough for it to miss the 9.0.
After the 9.0 is released, I will flip the switch with the following
change.
diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c
index 8455f48..926fe64 100644
--- a/sys/kern/imgact_elf.c
+++ b/sys/kern/imgact_elf.c
@@ -118,7 +118,12 @@ static int elf_legacy_coredump = 0;
SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump), CTLFLAG_RW,
elf_legacy_coredump, 0, );
-static int __elfN(nxstack) = 0;
+int __elfN(nxstack) =
+#if defined(__amd64__) || defined(__powerpc64__) /* both 64 and 32 bit */
Why leaving 32bits x86 CPU supporting the NX feature behind ?
- Arnaud
+ 1;
+#else
+ 0;
+#endif
SYSCTL_INT(__CONCAT(_kern_elf, __ELF_WORD_SIZE), OID_AUTO,
nxstack, CTLFLAG_RW, __elfN(nxstack), 0,
__XSTRING(__CONCAT(ELF, __ELF_WORD_SIZE)) : enable non-executable
stack);
diff --git a/sys/powerpc/aim/mmu_oea64.c b/sys/powerpc/aim/mmu_oea64.c
index 7500462..0e27351 100644
--- a/sys/powerpc/aim/mmu_oea64.c
+++ b/sys/powerpc/aim/mmu_oea64.c
@@ -1445,6 +1445,8 @@ moea64_uma_page_alloc(uma_zone_t zone, int bytes,
u_int8_t *flags, int wait)
return (void *)va;
}
+extern int elf32_nxstack;
+
void
moea64_init(mmu_t mmu)
{
@@ -1464,6 +1466,8 @@ moea64_init(mmu_t mmu)
uma_zone_set_allocf(moea64_mpvo_zone,moea64_uma_page_alloc);
}
+ elf32_nxstack = 1;
+
moea64_initialized = TRUE;
}
diff --git a/sys/powerpc/booke/machdep.c b/sys/powerpc/booke/machdep.c
index c2b5e6f..82a37e1 100644
--- a/sys/powerpc/booke/machdep.c
+++ b/sys/powerpc/booke/machdep.c
@@ -192,6 +192,8 @@ void print_kernel_section_addr(void);
void print_kenv(void);
u_int booke_init(uint32_t, uint32_t);
+extern int elf32_nxstack;
+
static void
cpu_e500_startup(void *dummy)
{
@@ -227,6 +229,9 @@ cpu_e500_startup(void *dummy)
/* Set up buffers, so they can be used to read disk labels. */
bufinit();
vm_pager_bufferinit();
+
+ /* Cpu supports execution permissions on the pages. */
+ elf32_nxstack = 1;
}
static char *
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] Enable nxstack by default
Hi, On Tue, Oct 18, 2011 at 11:44 AM, Garrett Cooper yaneg...@gmail.com wrote: On Tue, 18 Oct 2011, Arnaud Lacombe wrote: Hi, On Tue, Oct 18, 2011 at 5:07 AM, Kostik Belousov kostik...@gmail.com wrote: On Mon, Oct 17, 2011 at 09:30:56PM +0200, Oliver Pinter wrote: Hi all! I think, it's the time to enable the nxstack feature. Any comments, pros, cons? I dragged the change long enough for it to miss the 9.0. After the 9.0 is released, I will flip the switch with the following change. diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c index 8455f48..926fe64 100644 --- a/sys/kern/imgact_elf.c +++ b/sys/kern/imgact_elf.c @@ -118,7 +118,12 @@ static int elf_legacy_coredump = 0; SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump), CTLFLAG_RW, elf_legacy_coredump, 0, ); -static int __elfN(nxstack) = 0; +int __elfN(nxstack) = +#if defined(__amd64__) || defined(__powerpc64__) /* both 64 and 32 bit */ Why leaving 32bits x86 CPU supporting the NX feature behind ? Most likely because it was assumed that i386 doesn't fully support it. According to ye great Wikipedia, NX support didn't roll into i386 until Prescott, which was pretty late in the non-64-bit capable family of CPUs, as its successor -- Conroe -- was 64-bit. Intel detuned some of the early Dual Core Pentiums, e.g. the Yonahs to not talk 64-bit. Not sure about AMD. There are probably more details in binutils, gcc, etc, that I'm missing and Kostik can expound on. NX support is advertised in the cpuid flags, just add the logic to handle this interface. Kostik's patch is just incomplete, but he's got a commit bit so he can commit it as-is, as he will. If nonexec_stack becomes the default, it should be on every CPU supporting the feature, not just the low-hanging one. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] Enable nxstack by default
Hi, On Tue, Oct 18, 2011 at 12:53 PM, Oliver Pinter oliver.p...@gmail.com wrote: On 10/18/11, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Tue, Oct 18, 2011 at 11:44 AM, Garrett Cooper yaneg...@gmail.com wrote: On Tue, 18 Oct 2011, Arnaud Lacombe wrote: Hi, On Tue, Oct 18, 2011 at 5:07 AM, Kostik Belousov kostik...@gmail.com wrote: On Mon, Oct 17, 2011 at 09:30:56PM +0200, Oliver Pinter wrote: Hi all! I think, it's the time to enable the nxstack feature. Any comments, pros, cons? I dragged the change long enough for it to miss the 9.0. After the 9.0 is released, I will flip the switch with the following change. diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c index 8455f48..926fe64 100644 --- a/sys/kern/imgact_elf.c +++ b/sys/kern/imgact_elf.c @@ -118,7 +118,12 @@ static int elf_legacy_coredump = 0; SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump), CTLFLAG_RW, elf_legacy_coredump, 0, ); -static int __elfN(nxstack) = 0; +int __elfN(nxstack) = +#if defined(__amd64__) || defined(__powerpc64__) /* both 64 and 32 bit */ Why leaving 32bits x86 CPU supporting the NX feature behind ? Most likely because it was assumed that i386 doesn't fully support it. According to ye great Wikipedia, NX support didn't roll into i386 until Prescott, which was pretty late in the non-64-bit capable family of CPUs, as its successor -- Conroe -- was 64-bit. Intel detuned some of the early Dual Core Pentiums, e.g. the Yonahs to not talk 64-bit. Not sure about AMD. There are probably more details in binutils, gcc, etc, that I'm missing and Kostik can expound on. NX support is advertised in the cpuid flags, just add the logic to handle this interface. Kostik's patch is just incomplete, but he's got a commit bit so he can commit it as-is, as he will. If nonexec_stack becomes the default, it should be on every CPU supporting the feature, not just the low-hanging one. - Arnaud the NX detection code already implemented in i386, but this feature required PAE: yes, this is the conclusion I reached too. But this does not change the fact that the VM should know about that, and this is missing from Kostik's patch. I guess the first hunk should read: @@ -118,7 +118,12 @@ static int elf_legacy_coredump = 0; SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump), CTLFLAG_RW, elf_legacy_coredump, 0, ); -static int __elfN(nxstack) = 0; +int __elfN(nxstack) = +#if defined(PAE) || defined(__amd64__) || defined(__powerpc64__) /* both 64 and 32 bit */ + 1; +#else + 0; +#endif SYSCTL_INT(__CONCAT(_kern_elf, __ELF_WORD_SIZE), OID_AUTO, nxstack, CTLFLAG_RW, __elfN(nxstack), 0, __XSTRING(__CONCAT(ELF, __ELF_WORD_SIZE)) : enable non-executable stack); - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] Enable nxstack by default
Hi,
2011/10/18 Kostik Belousov kostik...@gmail.com:
On Tue, Oct 18, 2011 at 01:06:27PM -0400, Arnaud Lacombe wrote:
Hi,
On Tue, Oct 18, 2011 at 12:53 PM, Oliver Pinter oliver.p...@gmail.com
wrote:
On 10/18/11, Arnaud Lacombe lacom...@gmail.com wrote:
Hi,
On Tue, Oct 18, 2011 at 11:44 AM, Garrett Cooper yaneg...@gmail.com
wrote:
On Tue, 18 Oct 2011, Arnaud Lacombe wrote:
Hi,
On Tue, Oct 18, 2011 at 5:07 AM, Kostik Belousov kostik...@gmail.com
wrote:
On Mon, Oct 17, 2011 at 09:30:56PM +0200, Oliver Pinter wrote:
Hi all!
I think, it's the time to enable the nxstack feature. Any comments,
pros, cons?
I dragged the change long enough for it to miss the 9.0.
After the 9.0 is released, I will flip the switch with the following
change.
diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c
index 8455f48..926fe64 100644
--- a/sys/kern/imgact_elf.c
+++ b/sys/kern/imgact_elf.c
@@ -118,7 +118,12 @@ static int elf_legacy_coredump = 0;
SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump), CTLFLAG_RW,
elf_legacy_coredump, 0, );
-static int __elfN(nxstack) = 0;
+int __elfN(nxstack) =
+#if defined(__amd64__) || defined(__powerpc64__) /* both 64 and 32 bit
*/
Why leaving 32bits x86 CPU supporting the NX feature behind ?
Most likely because it was assumed that i386 doesn't fully support it.
According to ye great Wikipedia, NX support didn't roll into i386 until
Prescott, which was pretty late in the non-64-bit capable family of CPUs,
as
its successor -- Conroe -- was 64-bit. Intel detuned some of the early
Dual
Core Pentiums, e.g. the Yonahs to not talk 64-bit. Not sure about AMD.
There are probably more details in binutils, gcc, etc, that I'm missing
and
Kostik can expound on.
NX support is advertised in the cpuid flags, just add the logic to
handle this interface. Kostik's patch is just incomplete, but he's got
a commit bit so he can commit it as-is, as he will.
If nonexec_stack becomes the default, it should be on every CPU
supporting the feature, not just the low-hanging one.
- Arnaud
the NX detection code already implemented in i386, but this feature
required PAE:
yes, this is the conclusion I reached too. But this does not change
the fact that the VM should know about that, and this is missing from
Kostik's patch. I guess the first hunk should read:
@@ -118,7 +118,12 @@ static int elf_legacy_coredump = 0;
SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump), CTLFLAG_RW,
elf_legacy_coredump, 0, );
-static int __elfN(nxstack) = 0;
+int __elfN(nxstack) =
+#if defined(PAE) || defined(__amd64__) || defined(__powerpc64__) /*
both 64 and 32 bit */
+ 1;
+#else
+ 0;
+#endif
SYSCTL_INT(__CONCAT(_kern_elf, __ELF_WORD_SIZE), OID_AUTO,
nxstack, CTLFLAG_RW, __elfN(nxstack), 0,
__XSTRING(__CONCAT(ELF, __ELF_WORD_SIZE)) : enable non-executable
stack);
Your patch is not right, it will cause even more user confusion.
The presence of the PAE kernel does not imply that CPU supports nx.
Below is the updated patch that turns on nxstack by default for the PAE
kernels on NX-capable CPUs. Note that i386 usermode fully supports the
PT_GNU_STACK annotations and cares about not enabling nx on stack pages
unneccessary, but my main target was compat32 on amd64.
The fact that nxstack is not enabled by default does not prevent
administrator from manually enabling the feature.
Since you worried so much about PAE case, I sincerely expect that you
will test the change. Thank you in advance.
I will.
Btw, NetBSD has been going down the path of system unit test,
especially of kernel/userland interfaces, and already worked-out the
framework for that. Is that something FreeBSD would be interested in ?
Thanks,
- Arnaud
diff --git a/sys/i386/i386/initcpu.c b/sys/i386/i386/initcpu.c
index c2daf54..ec77adb 100644
--- a/sys/i386/i386/initcpu.c
+++ b/sys/i386/i386/initcpu.c
@@ -650,6 +650,8 @@ enable_sse(void)
#endif
}
+extern int elf32_nxstack;
+
void
initializecpu(void)
{
@@ -739,6 +741,7 @@ initializecpu(void)
msr = rdmsr(MSR_EFER) | EFER_NXE;
wrmsr(MSR_EFER, msr);
pg_nx = PG_NX;
+ elf32_nxstack = 1;
}
#endif
break;
diff --git a/sys/kern/imgact_elf.c b/sys/kern/imgact_elf.c
index 8455f48..926fe64 100644
--- a/sys/kern/imgact_elf.c
+++ b/sys/kern/imgact_elf.c
@@ -118,7 +118,12 @@ static int elf_legacy_coredump = 0;
SYSCTL_INT(_debug, OID_AUTO, __elfN(legacy_coredump), CTLFLAG_RW,
elf_legacy_coredump, 0, );
-static int __elfN(nxstack) = 0;
+int __elfN(nxstack) =
+#if defined(__amd64__) || defined(__powerpc64__) /* both 64 and 32 bit */
+ 1;
+#else
+ 0;
+#endif
SYSCTL_INT(__CONCAT(_kern_elf, __ELF_WORD_SIZE), OID_AUTO,
nxstack, CTLFLAG_RW, __elfN(nxstack), 0,
__XSTRING
Re: [RFC] Prepend timestamp in msgbuf
Hi, On Mon, Oct 17, 2011 at 2:01 PM, Alexander Best arun...@freebsd.org wrote: On Fri Oct 14 11, Arnaud Lacombe wrote: Hi, On Fri, Oct 14, 2011 at 8:52 AM, Nali Toja nalit...@gmail.com wrote: Alexander Best arun...@freebsd.org writes: On Fri Oct 14 11, Poul-Henning Kamp wrote: In message 20111014085609.ga3...@freebsd.org, Alexander Best writes: 1) would it be possible to prepend those timestamps to the actual console output and not only to the output of demsg? maybe via a sysctl toggle? The kernel does not know enough about timezones to emit anything but UTC timestamps. hmm ok. 2) my dmesg output contains a lot of these entries: 118 These are magic markers for syslogd(8) specifying priority. it would be nice, if their output could be turned off via a dmesg flag imo. 3) roughly the first 30 lines of my dmesg output have the timestamp [1.0]. would it be possible to have more accuracy there? No, because we don't know the time until we've found the RTC chip. maybe prepending the output with [??] instead of [1.0] would make more sense, so users knows that those timestamps are bogus. maybe the granularity of the timestamps could be limited to a static value? the following output doesn't really look pretty: [7.729516] 118/dev/ufs/varfs: clean, 879143 free (7407 frags, 108967 blocks, 0.7% fragmentation) [7.891512] 118Mounting local file systems:WARNING: TMPFS is considered to be a highly experimental feature in FreeBSD. [8.33519] . [9.440514] 118Setting hostname: otaku. [9.744516] wlan0: Ethernet address: 00:0f:b5:82:07:c8 [9.850516] 118Starting wpa_supplicant. [10.335514] 118Starting Network: lo0 ath0. so it would be nice, if trailing zeros got printed out, too. Why not make formatting similar to linux/xorg logs, e.g. [ 31.897] (**) Option XkbLayout us [ 31.897] (II) XINPUT: Adding extended input device default keyboard (type: KEYBOARD, id 7) [ 11485.404] (II) 3rd Button detected: disabling emulate3Button [ 0.00] Linux version 3.0-ARCH (tobias@T-POWA-LX) (gcc version 4.6.1 20110819 (prerelease) (GCC) ) #1 SMP PREEMPT Tue Aug 30 08:53:25 CEST 2011 [ 0.00] Command line: root=/dev/disk/by-uuid/625db1f5-9b51-4d2d-acb7-6726f4d7e199 ro [...] [ 15.096862] NET: Registered protocol family 10 [ 16.792594] [drm] nouveau :01:00.0: plugged DVI-I-2 [ 26.054186] eth0: no IPv6 routers present A way to convert those timestamps to localtime or time delta[1] post-mortem via dmesg(8) would be good, too. well, I do not care for the pretty side of the thing, however, this is just a matter length modifier in the string format; should be trivial to fix. cc -c -O -pipe -march=core2 -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -Wmissing-include-dirs -nostdinc -I. -I/usr/git-freebsd-head/sys -I/usr/git-freebsd-head/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-omit-frame-pointer -mno-sse -mcmodel=kernel -mno-red-zone -mno-mmx -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fformat-extensions -fdiagnostics-show-option -fstack-protector -Werror /usr/git-freebsd-head/sys/kern/subr_msgbuf.c cc1: warnings being treated as errors /usr/git-freebsd-head/sys/kern/subr_msgbuf.c: In function 'msgbuf_do_addchar': /usr/git-freebsd-head/sys/kern/subr_msgbuf.c:171: warning: format '%d' expects type 'int', but argument 4 has type 'time_t' [-Wformat] *** Error code 1 Stop in /usr/obj/usr/git-freebsd-head/sys/ARUNDEL. *** Error code 1 Stop in /usr/git-freebsd-head. *** Error code 1 Stop in /usr/git-freebsd-head. FreeBSD has no time_t PRI... macros in any machine/_inttypes.h, eventually cast it to `long'. Btw, I appreciate the very clear message of yours, no Hi, no signature, no idea what ARUNDEL is, and especially no details on which architecture you are attempting to build, which should be, I assume, LP64 ;-) - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] Prepend timestamp in msgbuf
Hi, On Mon, Oct 17, 2011 at 3:44 PM, Alexander Best arun...@freebsd.org wrote: On Mon Oct 17 11, Arnaud Lacombe wrote: Hi, On Mon, Oct 17, 2011 at 2:01 PM, Alexander Best arun...@freebsd.org wrote: On Fri Oct 14 11, Arnaud Lacombe wrote: Hi, On Fri, Oct 14, 2011 at 8:52 AM, Nali Toja nalit...@gmail.com wrote: Alexander Best arun...@freebsd.org writes: On Fri Oct 14 11, Poul-Henning Kamp wrote: In message 20111014085609.ga3...@freebsd.org, Alexander Best writes: 1) would it be possible to prepend those timestamps to the actual console output and not only to the output of demsg? maybe via a sysctl toggle? The kernel does not know enough about timezones to emit anything but UTC timestamps. hmm ok. 2) my dmesg output contains a lot of these entries: 118 These are magic markers for syslogd(8) specifying priority. it would be nice, if their output could be turned off via a dmesg flag imo. 3) roughly the first 30 lines of my dmesg output have the timestamp [1.0]. would it be possible to have more accuracy there? No, because we don't know the time until we've found the RTC chip. maybe prepending the output with [??] instead of [1.0] would make more sense, so users knows that those timestamps are bogus. maybe the granularity of the timestamps could be limited to a static value? the following output doesn't really look pretty: [7.729516] 118/dev/ufs/varfs: clean, 879143 free (7407 frags, 108967 blocks, 0.7% fragmentation) [7.891512] 118Mounting local file systems:WARNING: TMPFS is considered to be a highly experimental feature in FreeBSD. [8.33519] . [9.440514] 118Setting hostname: otaku. [9.744516] wlan0: Ethernet address: 00:0f:b5:82:07:c8 [9.850516] 118Starting wpa_supplicant. [10.335514] 118Starting Network: lo0 ath0. so it would be nice, if trailing zeros got printed out, too. Why not make formatting similar to linux/xorg logs, e.g. [ 31.897] (**) Option XkbLayout us [ 31.897] (II) XINPUT: Adding extended input device default keyboard (type: KEYBOARD, id 7) [ 11485.404] (II) 3rd Button detected: disabling emulate3Button [ 0.00] Linux version 3.0-ARCH (tobias@T-POWA-LX) (gcc version 4.6.1 20110819 (prerelease) (GCC) ) #1 SMP PREEMPT Tue Aug 30 08:53:25 CEST 2011 [ 0.00] Command line: root=/dev/disk/by-uuid/625db1f5-9b51-4d2d-acb7-6726f4d7e199 ro [...] [ 15.096862] NET: Registered protocol family 10 [ 16.792594] [drm] nouveau :01:00.0: plugged DVI-I-2 [ 26.054186] eth0: no IPv6 routers present A way to convert those timestamps to localtime or time delta[1] post-mortem via dmesg(8) would be good, too. well, I do not care for the pretty side of the thing, however, this is just a matter length modifier in the string format; should be trivial to fix. cc -c -O -pipe -march=core2 -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -Wmissing-include-dirs -nostdinc -I. -I/usr/git-freebsd-head/sys -I/usr/git-freebsd-head/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-omit-frame-pointer -mno-sse -mcmodel=kernel -mno-red-zone -mno-mmx -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fformat-extensions -fdiagnostics-show-option -fstack-protector -Werror /usr/git-freebsd-head/sys/kern/subr_msgbuf.c cc1: warnings being treated as errors /usr/git-freebsd-head/sys/kern/subr_msgbuf.c: In function 'msgbuf_do_addchar': /usr/git-freebsd-head/sys/kern/subr_msgbuf.c:171: warning: format '%d' expects type 'int', but argument 4 has type 'time_t' [-Wformat] *** Error code 1 Stop in /usr/obj/usr/git-freebsd-head/sys/ARUNDEL. *** Error code 1 Stop in /usr/git-freebsd-head. *** Error code 1 Stop in /usr/git-freebsd-head. FreeBSD has no time_t PRI... macros in any machine/_inttypes.h, eventually cast it to `long'. just wanted to tell you that your patch causes buildkernel to fail on certain archs (amd64 in my case) and that it appears you didn't run 'make universe-kernels' to check, whether your patch breaks anything. i'll leave the details to you. I have no interest testing architectures/kernels I have no interest in. Of course, this statement only apply as long as I have no full control over the future of the feature. - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [RFC] Prepend timestamp in msgbuf
Hi, On Mon, Oct 17, 2011 at 3:38 PM, Garrett Cooper yaneg...@gmail.com wrote: On Mon, Oct 17, 2011 at 12:27 PM, Arnaud Lacombe lacom...@gmail.com wrote: Hi, On Mon, Oct 17, 2011 at 2:01 PM, Alexander Best arun...@freebsd.org wrote: On Fri Oct 14 11, Arnaud Lacombe wrote: [...] cc -c -O -pipe -march=core2 -std=c99 -g -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -Wmissing-include-dirs -nostdinc -I. -I/usr/git-freebsd-head/sys -I/usr/git-freebsd-head/sys/contrib/altq -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -fno-omit-frame-pointer -mno-sse -mcmodel=kernel -mno-red-zone -mno-mmx -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fformat-extensions -fdiagnostics-show-option -fstack-protector -Werror /usr/git-freebsd-head/sys/kern/subr_msgbuf.c cc1: warnings being treated as errors /usr/git-freebsd-head/sys/kern/subr_msgbuf.c: In function 'msgbuf_do_addchar': /usr/git-freebsd-head/sys/kern/subr_msgbuf.c:171: warning: format '%d' expects type 'int', but argument 4 has type 'time_t' [-Wformat] *** Error code 1 Stop in /usr/obj/usr/git-freebsd-head/sys/ARUNDEL. *** Error code 1 Stop in /usr/git-freebsd-head. *** Error code 1 Stop in /usr/git-freebsd-head. FreeBSD has no time_t PRI... macros in any machine/_inttypes.h, eventually cast it to `long'. Btw, I appreciate the very clear message of yours, no Hi, no signature, no idea what ARUNDEL is, and especially no details on which architecture you are attempting to build, which should be, I assume, LP64 ;-) time_t maps to int32_t on i386 and int64_t on amd64 (at least), so you should be able to use %zd in the format string as the type is variable width depending on the architecture. make sense. Thanks, - Arnaud ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
[PATCH] Prepend timestamp in msgbuf
Hi folks,
You'll find hereafter a new version of the patch to add timestamp to msgbuf.
Broken out patches are available in the git repository at:
git://github.com/lacombar/freebsd.git master/topic/msgbuf-timestamp
Diff since RFC:
- build should be fixed on LP64
- micro-second field is now fixed width
Arnaud Lacombe (3):
msgbuf(4): convert `msg_needsnl' to a bit flag
msgbuf(4): add logic to prepend timestamp on new line
msgbuf(4): add a sysctl to toggle timestamp prepend
sys/kern/subr_msgbuf.c | 54 ---
sys/sys/msgbuf.h |4 ++-
2 files changed, 49 insertions(+), 9 deletions(-)
diff --git a/sys/kern/subr_msgbuf.c b/sys/kern/subr_msgbuf.c
index cd9c551..6b462eb 100644
--- a/sys/kern/subr_msgbuf.c
+++ b/sys/kern/subr_msgbuf.c
@@ -34,6 +34,7 @@
#include sys/lock.h
#include sys/mutex.h
#include sys/msgbuf.h
+#include sys/sysctl.h
/*
* Maximum number conversion buffer length: uintmax_t in base 2, plus
@@ -47,6 +48,13 @@
static u_int msgbuf_cksum(struct msgbuf *mbp);
/*
+ *
+ */
+static int msgbuf_show_timestamp = 1;
+SYSCTL_INT(_kern, OID_AUTO, msgbuf_show_timestamp, CTLFLAG_RW,
+msgbuf_show_timestamp, 0, Show timestamp in msgbuf);
+
+/*
* Initialize a message buffer of the specified size at the specified
* location. This also zeros the buffer area.
*/
@@ -60,7 +68,7 @@ msgbuf_init(struct msgbuf *mbp, void *ptr, int size)
msgbuf_clear(mbp);
mbp-msg_magic = MSG_MAGIC;
mbp-msg_lastpri = -1;
- mbp-msg_needsnl = 0;
+ mbp-msg_flags = 0;
bzero(mbp-msg_lock, sizeof(mbp-msg_lock));
mtx_init(mbp-msg_lock, msgbuf, NULL, MTX_SPIN);
}
@@ -95,7 +103,7 @@ msgbuf_reinit(struct msgbuf *mbp, void *ptr, int size)
mbp-msg_lastpri = -1;
/* Assume that the old message buffer didn't end in a newline. */
- mbp-msg_needsnl = 1;
+ mbp-msg_flags |= MSGBUF_NEEDNL;
bzero(mbp-msg_lock, sizeof(mbp-msg_lock));
mtx_init(mbp-msg_lock, msgbuf, NULL, MTX_SPIN);
}
@@ -134,7 +142,7 @@ msgbuf_getcount(struct msgbuf *mbp)
* The caller should hold the message buffer spinlock.
*/
static inline void
-msgbuf_do_addchar(struct msgbuf *mbp, u_int *seq, int c)
+__msgbuf_do_addchar(struct msgbuf *mbp, u_int *seq, int c)
{
u_int pos;
@@ -149,6 +157,35 @@ msgbuf_do_addchar(struct msgbuf *mbp, u_int *seq, int c)
*seq = MSGBUF_SEQNORM(mbp, *seq + 1);
}
+static inline void
+msgbuf_do_addchar(struct msgbuf *mbp, u_int *seq, int c)
+{
+
+ if (msgbuf_show_timestamp mbp-msg_flags MSGBUF_NEXT_NEW_LINE) {
+ char buf[32], *bufp;
+ struct timespec ts;
+ int err;
+
+ buf[0] = '\0';
+ getnanouptime(ts);
+ err = snprintf(buf, sizeof buf, [%zd.%.6ld] ,
+ ts.tv_sec, ts.tv_nsec / 1000);
+
+ bufp = buf;
+ while (*bufp != '\0') {
+ __msgbuf_do_addchar(mbp, seq, *bufp);
+ bufp++;
+ }
+
+ mbp-msg_flags = ~MSGBUF_NEXT_NEW_LINE;
+ }
+
+ __msgbuf_do_addchar(mbp, seq, c);
+
+ if (c == '\n')
+ mbp-msg_flags |= MSGBUF_NEXT_NEW_LINE;
+}
+
/*
* Append a character to a message buffer.
*/
@@ -207,10 +244,10 @@ msgbuf_addstr(struct msgbuf *mbp, int pri, char *str, int
filter_cr)
* did not end with a newline. If that is the case, we need to
* insert a newline before this string.
*/
- if (mbp-msg_lastpri != pri mbp-msg_needsnl != 0) {
+ if (mbp-msg_lastpri != pri (mbp-msg_flags MSGBUF_NEEDNL) != 0) {
msgbuf_do_addchar(mbp, seq, '\n');
- mbp-msg_needsnl = 0;
+ mbp-msg_flags = ~MSGBUF_NEEDNL;
}
for (i = 0; i len; i++) {
@@ -219,7 +256,7 @@ msgbuf_addstr(struct msgbuf *mbp, int pri, char *str, int
filter_cr)
* (and therefore prefix_len != 0), then we need a priority
* prefix for this line.
*/
- if (mbp-msg_needsnl == 0 prefix_len != 0) {
+ if ((mbp-msg_flags MSGBUF_NEEDNL) == 0 prefix_len != 0) {
int j;
for (j = 0; j prefix_len; j++)
@@ -242,9 +279,9 @@ msgbuf_addstr(struct msgbuf *mbp, int pri, char *str, int
filter_cr)
* we need to insert a new prefix or insert a newline later.
*/
if (str[i] == '\n')
- mbp-msg_needsnl = 0;
+ mbp-msg_flags = ~MSGBUF_NEEDNL;
else
- mbp-msg_needsnl = 1;
+ mbp-msg_flags |= MSGBUF_NEEDNL;
msgbuf_do_addchar(mbp, seq, str[i]);
}
@@ -395,3 +432,4 @@ msgbuf_copy(struct msgbuf *src, struct msgbuf *dst)
while ((c = msgbuf_getchar(src)) = 0)
msgbuf_addchar(dst, c
Re: [PATCH] Prepend timestamp in msgbuf
Hi,
On Mon, Oct 17, 2011 at 6:19 PM, Ed Schouten e...@80386.nl wrote:
Hi Arnaud!
* Arnaud Lacombe lacom...@gmail.com, 20111017 22:41:
+ buf[0] = '\0';
+ getnanouptime(ts);
+ err = snprintf(buf, sizeof buf, [%zd.%.6ld] ,
+ ts.tv_sec, ts.tv_nsec / 1000);
What's the use of buf[0] = '\0'? snprintf() will overwrite it anyway,
right?
leftover from previous debug I guess; fixed.
Also. please use %jd and cast ts.tv_sec to intmax_t. The size of
time_t and size_t are independent.
fixed.
As far as I know, you should be able
to use a 64-bit time_t on i386 by simply changing the typedef and
recompiling everything.
As long as you do not care about breaking the ABI, yes. But yet, the
kernel and the userland may not need to each have the same
representation of what `time_t' is, as long as they agree on the
interface.
+ bufp = buf;
+ while (*bufp != '\0') {
+ __msgbuf_do_addchar(mbp, seq, *bufp);
+ bufp++;
+ }
It would be nicer to write this as follows:
for (bufp = buf; *bufp != '\0'; bufp++)
__msgbuf_do_addchar(mbp, seq, *bufp);
fixed.
- int msg_needsnl; /* set when newline needed */
+ uint32_t msg_flags;
Why change this to uint32_t instead of leaving it the way it is (or
changing it to unsigned int)? Even though they are likely to be equal in
size, there is no reason why msg_flags must be 32 bits. :-)
made it `unsigned int'; I don't like playing with signed bit-field.
- Arnaud
--
Ed Schouten e...@80386.nl
WWW: http://80386.nl/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [PATCH] Prepend timestamp in msgbuf
Hi, On Mon, Oct 17, 2011 at 6:22 PM, Ed Schouten e...@80386.nl wrote: Ah, missed something. + getnanouptime(ts); + err = snprintf(buf, sizeof buf, [%zd.%.6ld] , + ts.tv_sec, ts.tv_nsec / 1000); It seems we also have a getmicrouptime(), which returns a struct timeval. fixed. Also a more general question: is it actually safe to call getnanouptime() here? This code gets executed from an arbitrary context, right? right, but getmicrouptime() is not doing much magic. Just reading a cached value, do an arithmetic conversion. I do not really see any unsafe part. - Arnaud -- Ed Schouten e...@80386.nl WWW: http://80386.nl/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
