I have a kid free afternoon I'll test it later. Tom
On 21 Jul 2017 9:20 pm, "Chris Mattmann" <mattm...@apache.org> wrote: > Thanks no worries. I just need 1 more +1 and I can release tomorrow… > > > > On 7/21/17, 1:15 PM, "BW" <w...@apache.org> wrote: > > Hi Chris. Haven't yet; deferred I guess. Sorry, on vacay.. > > On Fri, Jul 21, 2017 at 1:11 PM Chris Mattmann <mattm...@apache.org> > wrote: > > > Hi BW, have you tried RC #2? +1 still? > > > > Cheers, > > Chris > > > > > > > > > > On 7/19/17, 5:32 PM, "BW" <w...@apache.org> wrote: > > > > +1 > > > > On Wed, Jul 19, 2017 at 8:51 AM Chris Mattmann < > mattm...@apache.org> > > wrote: > > > > > Absolutely, here is a simple use case: > > > > > > If you look at CAS-PGE files, they typically include setup to > “stage > > files” > > > (already archived in File Manager) to the local job working > > directly. Many > > > times > > > this is just handled in the bash script (or Python or > whatever) that > > > CAS-PGE generates > > > itself. However it’s such a repetitive task and *part of* the > actual > > > workflow for running > > > a job, we added a simple interface to do so. See here: > > > > > > > > > > > https://raw.githubusercontent.com/apache/oodt/master/pge/ > src/test/resources/pge-config.xml > > > > > > And notice this block: > > > > > > <fileStaging dir="[StagingDir]" force="[ForceStaging]"> > > > <stageFiles metadataKey="InputFiles" /> > > > </fileStaging> > > > > > > What the above does is inspect the value of the InputFiles > metadata > > as > > > provided > > > to CAS-PGE (even if it’s a list) and for each of those files, > it > > “stages” > > > them (aka uses > > > the associated and declared DataTransferer for this CAS-PGE > > instance) to > > > the directory > > > defined by [StagingDir]. > > > > > > Cool huh? > > > > > > Cheers, > > > Chris > > > > > > > > > P.S. It doesn’t work now, but will shortly ;) > > > > > > > > > On 7/19/17, 8:43 AM, "Mallder, Valerie" < > valerie.mall...@jhuapl.edu> > > > wrote: > > > > > > Yes, adding a note to CHANGES.txt is fine. The fix to the > > fileStaging > > > feature of CAS-PGE sounds good, although I've never understood > what > > that > > > feature is supposed to do :) When you have some free time I > would > > love to > > > hear about what you are using it for. > > > > > > Thanks! > > > > > > > > > -----Original Message----- > > > From: Chris Mattmann [mailto:mattm...@apache.org] > > > Sent: Wednesday, July 19, 2017 11:37 AM > > > To: dev@oodt.apache.org > > > Subject: Re: [VOTE] Apache OODT 1.1 RC #1 > > > > > > Thank you Val! > > > > > > I may spin another RC not only because of the below (which > I > > think we > > > should just note in CHANGES.txt (I am happy to add a note on > it), > > but also > > > b/c I found an interesting bug while testing OODT 1.1 RC #1 > with > > DRAT – the > > > bug relates to using fileStaging – it doesn’t explicitly set a > > > Product.PRODUCT_STRUCTURE* and as so gets an NPE if you try > and use > > it. I > > > never encountered this before b/c I never used OODT’s native > file > > staging > > > support in CAS-PGE. It is a simple fix, and I think will add a > lot > > of value > > > (fileStaging is inoperable without it in CAS-PGE). > > > > > > Due to that, I will spin another RC shortly. > > > > > > Thank you! > > > > > > Cheers, > > > Chris > > > > > > > > > > > > > > > On 7/19/17, 7:55 AM, "Mallder, Valerie" < > > valerie.mall...@jhuapl.edu> > > > wrote: > > > > > > +1 > > > > > > I downloaded the source and built it with "mvn package > > -skipTests" > > > and it built ok. I did a system test by running some data > through my > > > pipeline that uses cas-workflow-1.1, cas-filemgr-1.1 and > cas-pge-1.1 > > and > > > everything worked as I expected. And it even helped find some > > things that > > > I needed to tweak in my test environment :) > > > > > > The only problem I should note is that had to delete > the > > workflow > > > repository directory and allow cas-workflow-1.1 to create a > > repository from > > > scratch. cas-workflow-1.1 reported an incompatibility with > workflow > > > repository that was created with cas-workflow-0.10 and has > been used > > with > > > cas-workflow-1.0 for a while now. (Also note that, if the > repository > > > directory exists but it is empty, workflow manager will crash. > So, I > > had to > > > delete the repository directory itself rather than just delete > the > > contents > > > within it.) > > > > > > The error I received is below. It looks like the > change from > > > lucene-core-2.0.0.jar to the lucene-core-6.1.0.jar caused this > > > incompatibility between cas-workflow-1.0 and cas-workflow-1.1. > > Deleting my > > > pipeline's workflow respository and allowing cas-workflow-1.1 > to > > create a > > > new one was not a problem for me, but I can't speak for > everyone. > > > > > > Jul 19, 2017 10:45:02 AM > > > > > org.apache.oodt.cas.workflow.instrepo.LuceneWorkflowInstanceReposito > ry > > > addWorkflowInstanceToCatalog > > > WARNING: Unable to index workflow instance: > > > [d8dad567-6c90-11e7-9e4e-7574e3a12106]: Message: Format > version is > > not > > > supported (resource > > > > > BufferedChecksumIndexInput(MMapIndexInput(path="/project/ > oodt/dev/jedi/data/workflow/slothrop/segments"))): > > > -1 (needs to be between 1071082519 and 1071082519). This > version of > > Lucene > > > only supports indexes created with release 5.0 and later. > > > java.lang.NullPointerException > > > > > > Val > > > > > > > > > > > > -----Original Message----- > > > From: Chris Mattmann [mailto:mattm...@apache.org] > > > Sent: Tuesday, July 18, 2017 6:54 PM > > > To: dev@oodt.apache.org > > > Subject: [VOTE] Apache OODT 1.1 RC #1 > > > > > > Hi Folks, > > > > > > I have posted a 1st release candidate for the Apache > OODT 1.1 > > > release. The source code is at: > > > > > > https://dist.apache.org/repos/dist/dev/oodt/ > > > > > > For more detailed information, see the included > CHANGES.txt > > file > > > for details on release contents and latest changes. The > release was > > made > > > using the OODT release process, documented on the Wiki here: > > > > > > > > https://cwiki.apache.org/confluence/display/OODT/Release+Process > > > > > > The release was made from the OODT 1.1 tag at: > > > > > > https://github.com/apache/oodt/tree/1.1 > > > > > > A staged Maven repository is available at: > > > > > > > > > > > https://repository.apache.org/content/repositories/ > orgapacheoodt-1012/ > > > > > > Please vote on releasing these packages as Apache OODT > 1.1. > > The > > > vote is open for at least the next 72 hours. > > > > > > Only votes from OODT PMC are binding, but folks are > welcome > > to > > > check the release candidate and voice their approval or > disapproval. > > The > > > vote passes if at least three binding +1 votes are cast. > > > > > > [ ] +1 Release the packages as Apache OODT 1.1 > > > > > > [ ] -1 Do not release the packages because... > > > > > > Thanks! > > > > > > Chris Mattmann > > > > > > P.S. Here is my +1. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
