> how did you know where to look,
> what tools do you use to navigate the lower levels of linux,
> and can you recommend a "gentle" introduction to lower levels
> within the kernel?
Tools: I used little known but the best programmer's editor I know:
SourceInsight (http://www.sourceinsight.com/). It might not be kosher
for some (windows only) but it's just awesome. It's highly oriented
towards C/C++/Java but for coding in those languages it's simply the
best. The biggest thing about SI is its ability to parse C/C++ code so
it knows which string is a variable, class member, enum etc. (and it's
not a primitive regexp parsing as in Emacs or most other tools good only
for syntax highligting, I believe they have pretty much full C/C++
parser). SI really shines when you have to navigate through big programs
(especially the code you don't know). This is something I tend to do a
lot and I can't imagine doing that without SI. They have a trial
version, play with it. I belong to those 42%
(http://www.sourceinsight.com/eval.htm) that say that SI "Greatly
improved productivity". Highly recommend.
How did I know: I created a SourceInsight project for each kernel, SI
parsed and indexed all files in a few minutes, I looked for places where
EWOULDBLOCK is used in the whole tree, not many of them but noticed that
EAGAIN is a synonim, so looked for EAGAIN, used in many places. I
figured out that the function would be called somewhat *recv*, I spent
some time in af_unix.c and af_inet.c but that wasn't it, found tcp.c,
read it for a while and decided that that's it.
In other words: guess work that wouldn't be possible (or at least not so
easy) without SI.
Introduction: I don't know, this is the first time I've read kernel
source and I haven't read any intros. There are some books about kernel,
both on the net and on paper. But if you want just to answer a very
specific question (as in this EWOULDBLOCK example) SourceInsight +
sources are your best bet.