Thanks Jacek for the info.
I agree with those use cases. As I indicated, astronomers often want
to complete their analyses with the same versioning of the calibration
pipeline and algorithms. Similarly, they may wish to go back to that
version (earlier release) in order to verify or reproduce some previously
published results. Thus, people will want the older versions, but
(as you say) we have to determine what are our legitimate QoS (Quality
of Service) obligations in this regard.
- Kirk
> Date: Tue, 11 Jul 2006 16:17:17 -0700
> From: Jacek Becla <[EMAIL PROTECTED]>
> To: Kirk Borne <[EMAIL PROTECTED]>,
LSST Data Management <[email protected]>
> Subject: Re: [LSST-data] disk space for old releases
>
> Kirk
>
> We will carry forward between releases all the Sources (detections),
> but we are not going to carry between releases all versions of all
> objects (objects = star or galaxy in deep/coadded catalog). So in
> practice, if you base your publication on Object table from DR 2,
> and we have on disk DR 3 and 4, in order to reproduce your results
> you will need to stage data from DR2.
>
> Another use case: students/astronomers who want to stay with
> a given release and do their analysis on a relatively small data
> sample for an extended period of time.
>
> The answer might be that whoever wants to get data from old
> releases need to find a space at his local institute and stage it
> there, I don't know. I do know we need to take care of this issue.
> Hmm, let's talk about it tomorrow.
>
> Jacek
>
>
>
>
> Kirk Borne wrote:
> > I guess I am not clear on what would be contained in the older releases.
> >
> > In some (most?) projects, the newer releases supercede (and
> > include) the contents of the older releases. I suppose the
> > older releases do include unique calibrated data products
> > that were calibrated under some prior version of the data
> > processing/calibration algoriths. In that case, I can see
> > some utility and value in having the older releases available.
> > However, having these releases instantly accessible on spinning
> > disks (versus archival backup media) is another issue.
> >
> > - Kirk
> >
> >
> >
> >>Date: Tue, 11 Jul 2006 15:56:10 -0700
> >>From: Jacek Becla <[EMAIL PROTECTED]>
> >>Subject: [LSST-data] disk space for old releases
> >>To: LSST Data Management <[email protected]>
> >>
> >>Keywords: DataAccWG
> >>
> >>Hello,
> >>
> >>In our current disk storage estimates we are assuming we will need
> >>disk space for 2 most recent releases and unreleased catalog.
> >>But what about the disk space for older releases that some people might
> >>want to stage in from tape? That is not included in the estimates.
> >>How much should we reserve for that? Equivalent of size of the most
> >>recent release?
> >>
> >>BTW, if we end up being limited by disk IO and not space, we will
> >>get that "space for free" anyway, BUT... it is important to get
> >>an idea how much disk space we will need because disk io pushes us
> >>towards smaller disks (which are faster, have better seek time),
> >>so we can end up in a situation where required number of
> >>small disks does not give us enough disk space (I went through
> >>the numbers with Don and that can happen). I'm in the process
> >>of building a model for that.
> >>
> >>
> >>Jacek
_______________________________________________
LSST-data mailing list
[email protected]
http://www.lsstmail.org/mailman/listinfo/lsst-data
