Tech Assess concall June 12, 2006 The discussion centered around the expected needs for a software development system that would be used by pipeline and middleware developers. We discussed the LSST code development environment. - Codes are written by individuals at their own institutions on their local computers. One can easily imagine that this involves around a 100 astronomers/programmers. - As the codes mature, they will be collected on the LSST development system. - Since the individuals use whatever platform they chose to write their code, the first step of the integration is to rebuild their code on the development system so all codes share a common system environment of operating system, compilers, libraries, and file system. The developers tests their codes again and perhaps gain a very crude idea of the code performance on the new platform. The development system has several ÔloginÕ nodes for this work. Multiple developers use these login nodes simultaneously. - On a very regular basis, perhaps nightly, the existing integrated software is built and tested to the extent possible. Individual developers are responsible for providing the tests. It may be desirable to have 2 builds - one for the more stable environment and another for a much more immature integration. - We discussed how to test those codes which are inherently parallel. Perhaps we carve out a small ÔdebugÕ partition of multi-processor nodes which developers can use to test parallel codes. Debug policies are set with some maximum runtime so everyone gets a chance at the debug partition without an unreasonable delay. These debug nodes are allocated to one user at a time. - The majority of the compute nodes test the correct operation of much larger scale applications and gain a more precise measurement of their performance. Again, these nodes are dedicated to a single user at a time. Let me call this the ÔlargeÕ partition. - Both parallel environments would be used by individual programmers and the system integration team tying pipelines together with middleware. - There will be occasions when the system integration team will need nearly the entire development system to measure the performance of the existing DM pipelines comprising computing, databases, and middleware. - We discussed the need to have at least 2 and potentially 3 separate file systems of increasingly greater size so that preliminary testing does not impact performance testing on the ÔlargeÕ partition. These individual file systems would be on separate disks using separate host disk controllers. - To run the largest possible software functional and performance test, we will need some mechanism to connect all the file system servers to the large disk farm. - Another work item is defining an acceptance test for the hardware at the development center as well as the summit/base/archive sites. Passing a stringent hardware acceptance test means we have reliable hardware. Suppliers are motivated to meet our criteria by including the definition of the acceptance test in the contract with the provision that they are not paid until they pass it. - An equivalent acceptance test should be defined for our integrated software environment so the project team can track our progress and understand what we would be ÔdeliveringÕ to them. - We will continue this quite fascinating and enlightening discussion at the next tech assess call on June 19. - Additional discussions outside of the concall concern the floor space, power and cooling needs of the summit and base computers. Victor K. is expected to be in La Serena during the week of July 17. I am trying to arrange my schedule to be there at the same time and also bring along one of LLNLÕs senior facility managers who is an expert in specify, installing, and debugging large computer rooms.
Don
_______________________________________________ LSST-data mailing list [email protected] http://www.lsstmail.org/mailman/listinfo/lsst-data
