I think this is subjective. We may want logs to be separated by experiment, workflows, applications, resources, users, type of resources (all ec2 logs), and so forth.
The trend seems to be log verbosely (with proper tags as Terri listed and may be more fields), rotate aggressively, and let a downstream component on a non-critical path stream all these into HDFS like stores and allow processing on them. An example would be funnel all airavata log in last hour, and the quires can be: search for a particular experiment id, search for all experiment which ran on trestles, search for all experiments which were executed by a particular gfac instance, search by user, inputs and so on. Suresh On Aug 28, 2014, at 10:53 AM, Marlon Pierce <[email protected]> wrote: > I'm wondering if it would be a good idea to create a separate log file for > each experiment. We current direct everything to airavata.log and > airavata-server.out. As a consequence, log entries for different submissions > get interleaved, which makes tracing a particular experiment's life cycle > difficult. > > Note there is a 32,000 file limit per directory in Linux by default (unless > my knowledge is obsolete), so we'd need to take this into account. > > Alternatively, we could do a better job of labeling the entries in > airavata.log so that it was clear which experiment is associated with the > entry. > > Other suggestions? > > Marlon >
