Hi BW, have you tried RC #2? +1 still?
Cheers,
Chris
On 7/19/17, 5:32 PM, "BW" <w...@apache.org> wrote:
+1
On Wed, Jul 19, 2017 at 8:51 AM Chris Mattmann <mattm...@apache.org> wrote:
> Absolutely, here is a simple use case:
>
> If you look at CAS-PGE files, they typically include setup to “stage
files”
> (already archived in File Manager) to the local job working directly. Many
> times
> this is just handled in the bash script (or Python or whatever) that
> CAS-PGE generates
> itself. However it’s such a repetitive task and *part of* the actual
> workflow for running
> a job, we added a simple interface to do so. See here:
>
>
>
https://raw.githubusercontent.com/apache/oodt/master/pge/src/test/resources/pge-config.xml
>
> And notice this block:
>
> <fileStaging dir="[StagingDir]" force="[ForceStaging]">
> <stageFiles metadataKey="InputFiles" />
> </fileStaging>
>
> What the above does is inspect the value of the InputFiles metadata as
> provided
> to CAS-PGE (even if it’s a list) and for each of those files, it “stages”
> them (aka uses
> the associated and declared DataTransferer for this CAS-PGE instance) to
> the directory
> defined by [StagingDir].
>
> Cool huh?
>
> Cheers,
> Chris
>
>
> P.S. It doesn’t work now, but will shortly ;)
>
>
> On 7/19/17, 8:43 AM, "Mallder, Valerie" <valerie.mall...@jhuapl.edu>
> wrote:
>
> Yes, adding a note to CHANGES.txt is fine. The fix to the fileStaging
> feature of CAS-PGE sounds good, although I've never understood what that
> feature is supposed to do :) When you have some free time I would love to
> hear about what you are using it for.
>
> Thanks!
>
>
> -----Original Message-----
> From: Chris Mattmann [mailto:mattm...@apache.org]
> Sent: Wednesday, July 19, 2017 11:37 AM
> To: dev@oodt.apache.org
> Subject: Re: [VOTE] Apache OODT 1.1 RC #1
>
> Thank you Val!
>
> I may spin another RC not only because of the below (which I think we
> should just note in CHANGES.txt (I am happy to add a note on it), but also
> b/c I found an interesting bug while testing OODT 1.1 RC #1 with DRAT –
the
> bug relates to using fileStaging – it doesn’t explicitly set a
> Product.PRODUCT_STRUCTURE* and as so gets an NPE if you try and use it. I
> never encountered this before b/c I never used OODT’s native file staging
> support in CAS-PGE. It is a simple fix, and I think will add a lot of
value
> (fileStaging is inoperable without it in CAS-PGE).
>
> Due to that, I will spin another RC shortly.
>
> Thank you!
>
> Cheers,
> Chris
>
>
>
>
> On 7/19/17, 7:55 AM, "Mallder, Valerie" <valerie.mall...@jhuapl.edu>
> wrote:
>
> +1
>
> I downloaded the source and built it with "mvn package -skipTests"
> and it built ok. I did a system test by running some data through my
> pipeline that uses cas-workflow-1.1, cas-filemgr-1.1 and cas-pge-1.1 and
> everything worked as I expected. And it even helped find some things that
> I needed to tweak in my test environment :)
>
> The only problem I should note is that had to delete the workflow
> repository directory and allow cas-workflow-1.1 to create a repository
from
> scratch. cas-workflow-1.1 reported an incompatibility with workflow
> repository that was created with cas-workflow-0.10 and has been used with
> cas-workflow-1.0 for a while now. (Also note that, if the repository
> directory exists but it is empty, workflow manager will crash. So, I had
to
> delete the repository directory itself rather than just delete the
contents
> within it.)
>
> The error I received is below. It looks like the change from
> lucene-core-2.0.0.jar to the lucene-core-6.1.0.jar caused this
> incompatibility between cas-workflow-1.0 and cas-workflow-1.1. Deleting my
> pipeline's workflow respository and allowing cas-workflow-1.1 to create a
> new one was not a problem for me, but I can't speak for everyone.
>
> Jul 19, 2017 10:45:02 AM
> org.apache.oodt.cas.workflow.instrepo.LuceneWorkflowInstanceRepository
> addWorkflowInstanceToCatalog
> WARNING: Unable to index workflow instance:
> [d8dad567-6c90-11e7-9e4e-7574e3a12106]: Message: Format version is not
> supported (resource
>
BufferedChecksumIndexInput(MMapIndexInput(path="/project/oodt/dev/jedi/data/workflow/slothrop/segments"))):
> -1 (needs to be between 1071082519 and 1071082519). This version of Lucene
> only supports indexes created with release 5.0 and later.
> java.lang.NullPointerException
>
> Val
>
>
>
> -----Original Message-----
> From: Chris Mattmann [mailto:mattm...@apache.org]
> Sent: Tuesday, July 18, 2017 6:54 PM
> To: dev@oodt.apache.org
> Subject: [VOTE] Apache OODT 1.1 RC #1
>
> Hi Folks,
>
> I have posted a 1st release candidate for the Apache OODT 1.1
> release. The source code is at:
>
> https://dist.apache.org/repos/dist/dev/oodt/
>
> For more detailed information, see the included CHANGES.txt file
> for details on release contents and latest changes. The release was made
> using the OODT release process, documented on the Wiki here:
>
> https://cwiki.apache.org/confluence/display/OODT/Release+Process
>
> The release was made from the OODT 1.1 tag at:
>
> https://github.com/apache/oodt/tree/1.1
>
> A staged Maven repository is available at:
>
>
> https://repository.apache.org/content/repositories/orgapacheoodt-1012/
>
> Please vote on releasing these packages as Apache OODT 1.1. The
> vote is open for at least the next 72 hours.
>
> Only votes from OODT PMC are binding, but folks are welcome to
> check the release candidate and voice their approval or disapproval. The
> vote passes if at least three binding +1 votes are cast.
>
> [ ] +1 Release the packages as Apache OODT 1.1
>
> [ ] -1 Do not release the packages because...
>
> Thanks!
>
> Chris Mattmann
>
> P.S. Here is my +1.
>
>
>
>
>
>
>
>
>
>
>
>
