Hi Lee,
On Mon, Dec 23, 2013 at 12:19 AM, Lee Winter <[email protected]>wrote:
> On Sun, Dec 22, 2013 at 10:41 PM, Mike Shal <[email protected]> wrote:
> > Hi Lee,
> >
> > On Fri, Dec 20, 2013 at 10:23 PM, Lee Winter <[email protected]>
> > wrote:
> >>
> >> Recently I've been trying to use debug tools to understand what
> >> happens to a software tool that is invoked by tup. But the
> >> .tup/mnt/@tupjob-#/ directories have been impenetrable.
> >>
> >> In fuse_fs.c I've disabled the pgid check so that other process groups
> >> should be able to access the fuse directories in the same way the
> >> tup-launched software tool accesses those directories.
> >
> >
> > How did you disable the pgid check? I tried to just do a 'return 0;' in
> > context_check() and I am able to access the .tup/mnt directory. Doing 'ls
> > .tup/mnt' should show the root file-system, as should 'ls
> > .tup/mnt/@tupjob-X'.
>
> Exactly. When context_check() always returns zero and the build step
> takes forever then when I use ls -al I see:
>
> in .tup/
> all the typical .tup files and directories
> mnt/
>
> in .tup/mnt/
> copies of all of the root files and directories but the link
> counts differ slightly
> NO SIGN OF @tupjob-#
>
> I cannot detect @tupjob-#, (ls, df, etc.) but I can cd to it!
>
This discrepancy is because you can do weird things with FUSE, which tup is
apparently doing. Using 'ls' will call into tup_fs_readdir(), and the FUSE
fs can report anything it wants to. With 'cd' on the other hand, it just
calls tup_fs_getattr() and tup_fs_access() to see if the given path is
indeed a directory. Since tup_fs_readdir() doesn't do anything special to
report the @tupjob directories, ls won't see them. But since getattr/access
*are* doing special things (by trimming the @tupjob directory from the path
before fstatat/faccessat), we can still cd into it.
>
> in tupjob-#/
> duplicate of the contents of mnt/
>
> My prompt shows .../.tup/mnt/@tupjob-#. And pwd show the same. So I
> think the cd worked.
>
> I expected to find only the build directories under
> .../tup/mnt/@tupjob-#/, so seeing the root files there was a bit
> disconcerting. Why are they in both .tup/mnt/ and in
> .tup/mnt/tupjob-#/ ?
>
I believe this is a historical footnote. Originally the @tupjob marker was
later in the path, so from the root we had something like:
/home/marf/project/.tup/mnt/home/marf/project/@tupjob-12/foo.c
But when support was added to track dependencies on files outside of the
tup hierarchy, the marker was moved:
/home/marf/project/.tup/mnt/@tupjob-12/home/marf/project/foo.c
Accessing /home/marf/.tup/mnt/ just gets translated to '/' in both the old
and new cases. In the new case we also strip the @tupjob-N marker, so
/home/marf/.tup/mnt/@tupjob-12/ also gets translated to '/'. This is why
you see the same contents in both directories.
>
> My /home/ is symlinked to /var/home/. So cding to /home/ within
> .tup/mnt/@tupjob-#/ fails with "no such directory". But cding to
> .tup/mnt/@tupjob-#//var/home works. And from there I can cd down to
> the build step working directory.
>
> The issues are:
> 1. the invisibility of @tupjob-# within .tup/mnt/
> 2. the presence of all the root files within /tup/mnt/@tupjob-#/
> 3. how to get fuse to follow symlinks so that they are safe to use
> within the build tree.
>
Can you try the attached patch for 1) and 2) and let me know if it helps?
It doesn't affect any test cases, but changes the behavior of 'ls' in
.tup/mnt to list the @tupjob markers (it's before a context_check() since
that doesn't matter here).
Do you have a specific issue with 3) in a normal build usage? (Ie: for
something not involving disabling the context check and peeking into the
file-system from outside of tup).
>
> > What are you ultimately trying to determine? Aside from the @tupjob paths
> > appearing in pwd, is there something not behaving as you would expect?
>
> I am trying to understand the pgids seen by context_check() when other
> programs access the build directory tree. Some of my debug tools do
> not fit into a build step, so I need to run the build part way, pause
> it, and then run the tools.
>
> Eventually I hope to enhance context_check() so that it maintains a
> (small) dynamic list of authorized pgids so external tools can be
> used.
>
> I am working on Debian 7.2.0 and using the tup sources from
> approximately 2013-12-01.
>
>
>
The reason for the context_check() is to make sure things like file-system
scanners don't go into the @tupjob directories and start opening or
statting files, since they would incorrectly be added to the dependencies
of the corresponding job. I'm not sure how to reconcile that with how you
are using your debugging tools, but if you think of something let me know!
-Mike
--
--
tup-users mailing list
email: [email protected]
unsubscribe: [email protected]
options: http://groups.google.com/group/tup-users?hl=en
---
You received this message because you are subscribed to the Google Groups
"tup-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
diff --git a/src/tup/server/fuse_fs.c b/src/tup/server/fuse_fs.c
index 230320d..a6b8254 100644
--- a/src/tup/server/fuse_fs.c
+++ b/src/tup/server/fuse_fs.c
@@ -632,6 +632,25 @@ static int tup_fs_readdir(const char *path, void *buf, fuse_fill_dir_t filler,
(void) offset;
(void) fi;
+ if(strcmp(path, "/") == 0) {
+ struct thread_tree *tht;
+ int rc = 0;
+ char tupjob[100];
+ if(filler(buf, ".", NULL, 0))
+ return -1;
+ if(filler(buf, "..", NULL, 0))
+ return -1;
+ pthread_mutex_lock(&troot.lock);
+ RB_FOREACH(tht, thread_entries, &troot.root) {
+ snprintf(tupjob, sizeof(tupjob), TUP_JOB "%i", tht->id);
+ tupjob[sizeof(tupjob)-1] = 0;
+ if(filler(buf, tupjob, NULL, 0))
+ rc = -1;
+ }
+ pthread_mutex_unlock(&troot.lock);
+ return rc;
+ }
+
if(context_check() < 0)
return -EPERM;
