Notes from Database-DC1 telecon
Thursday 27-July
Architecture:
The base camp and the archive center will each run 3 servers:
1 Ingest Server - database to load new nightly data into
2 Archive Servers - database to support queries from portal
The ingest server is separate from the archive servers to prevent
competition between ingest and queries. The 2 archive servers
is for load balancing. We are modeling this as if the data is
partitioned strategically (by sky position). These 2 archive servers
will contain both pre-existing data (e.g. astrometry catalog) and
newly ingested data from this night.
Note: There is some question about how effective partitioning will be
in this case.
Data from ingest server wil be replicated to archive servers using the
usual master-slave mechanisms supported by MySQL.
Issues:
1. Ingest Format -- to be discussed offline
2. How does a pipeline node know which archive server to query?
-- for DC1, nodes will be assigned a server has part of policy
file read at startup.
Testing:
We will need a full environment to test the ingest service and queries
from the pipeline using MySQL v5.0. To that end:
o Jacek: tries an install of MySQL installation in user space,
(tg scratch disk as himself) notes any special instructions for
doing so. He send instructions to Ray & Steve.
o Steve creates pacman script for MySQL v5.0 using instructions.
o Ray and Steve establish a testing sandbox on teragrid, probably
under /usr/projects/LSST. We will use group permission with our
group name (lso) to enable access by everyone. (This will
require that everyone set umask to 002.)
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
