Nathan,
Can you please do a configure make on your home directory system?
It could be since Lustre is a parallel filesystem designed for high
bandwidth IO from a parallel computer it is just bad for compiles etc.
Barry
https://www.olcf.ornl.gov/support/getting-started/
home areas
Home areas/directories are provided on a Network File System (NFS). The user
home areas are by default accessible only to the owning user. Similarly the
project home areas are accessible by only member of the project. The areas are
backed-up but available space is limited. Because space is limited, each area
has a quota. Users should store small source code, scripts, and other similar
items in the area. Users should not store large job output or input in the
area. Job I/O should be performed in the system’s temporary work area.
work areas
Temporary work directories are provided to each user and project on Lustre file
systems. Similar to the home areas, by default, the user and project areas are
accessible to the user and project members. The areas provide a large amount of
storage, but are not backed-up. In general, job I/O performance will be faster
in the lustre areas than the NFS mounted home areas. The temporary work areas
are regularly purged of data that has not been recently accessed, because of
this all needed data should be backed-up to the HPSS.
> On Feb 26, 2015, at 10:00 PM, Nathan Collier <[email protected]>
> wrote:
>
> > They never told you to do compiles on particular file systems or anything?
> > For example did they ever say DON'T compile on the lustre file system?
>
> I have not received any guidance about where to or not to compile. A quick
> perusing of the user guide doesn't make that clear to me. If it isn't already
> obvious, I am far from a Titan expert :)
>
> Do you know someone here at ORNL who I can talk to to make sure I am not
> missing something silly?
>
> > Perhaps the 52m 'sys' time could be luster overhead
>
> It could be, I have heard a lot of complaints about this from people who need
> to write output or read from files in their applications.
>
> Nate
>
>
> On Thu, Feb 26, 2015 at 10:36 PM, Satish Balay <[email protected]> wrote:
> On Thu, 26 Feb 2015, Barry Smith wrote:
>
> >
> > > On Feb 26, 2015, at 9:16 PM, Nathan Collier <[email protected]>
> > > wrote:
> > >
> > > Barry,
> > >
> > > > Do you know where /tmp is on the system? Presumably it is fast?
> > >
> > > I am not sure about that. There is a /tmp but I am not sure what it is or
> > > if/how I can use it.
> >
> > They never told you to do compiles on particular file systems or
> > anything? For example did they ever say DON'T compile on the lustre file
> > system?
>
> Ah - I misunderstood earlier. You'd like to have a petsc clone in /tmp
> - and do the build there - and compare the difference [with the
> current luster build].
>
> Perhaps the 52m 'sys' time could be luster overhead - and that woud
> disappear with a petsc build in /tmp. [and could be much faster].
>
> Satish
>
