Re: Remote data transfer

2014-08-18 Thread Mattmann, Chris A (3980)
Hi Etienne, Thanks. The Push Pull system is a way to pull down remote or ancillary files usually *ahead* of file manager ingestion, since the crawler really doesn't have a protocol layer to mitigate remote content. The typical use case if you use Push Pull is: 1. Model remote/ancillary files on

RE: Remote data transfer

2014-08-18 Thread Etienne Koen
Hi Tomas and all, I came across the push/pull tutorial on https://cwiki.apache.org/confluence/display/OODT/OODT+Push-Pull+User+Guide. Would this guide be more appropriate to download files that have been archived by the file manager and represent a typical user scenario? Regards Etienne

Re: Remote data transfer

2014-08-18 Thread Thomas Bennett
Hi Etienne, For a good description of the push pull architecture checkout the following web link: http://oodt.apache.org/components/maven/pushpull/development/developer.html I think that this represents a good method to use due to its extensibility. I will at some point start implementing this

Re: Remote data transfer

2014-08-18 Thread Thomas Bennett
Thanks Chris. Just to add to the conversation - what protocols are currently supported? I've seen scp, FTP and http. Also Amazon S3? On Monday, August 18, 2014, Mattmann, Chris A (3980) chris.a.mattm...@jpl.nasa.gov wrote: Hi Etienne, Thanks. The Push Pull system is a way to pull down

Re: Remote data transfer

2014-08-18 Thread Tom Barber
I promise to finish my Pentaho PDI plugins as well at some point, then you can all slurp and transform and ingest from pretty much anywhere. Tom On 18/08/14 19:39, Thomas Bennett wrote: Thanks Chris. Just to add to the conversation - what protocols are currently supported? I've seen scp,

Re: Remote data transfer

2014-08-18 Thread Mattmann, Chris A (3980)
BOOM, thanks Tom. ++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email:

Re: Remote data transfer

2014-08-18 Thread Mattmann, Chris A (3980)
Hi Tom, Great question! By default, all of the protocols support here in the cas-protocol module of Apache OODT: http://svn.apache.org/repos/asf/oodt/trunk/protocol/ * ftp * http(s) * imaps * sftp Note that there is an Amazon S3 data transfer module in the File Manager, but not explicitly in

Re: Review Request 24539: Fix Resource Manager in Trunk.

2014-08-18 Thread BW
Is there not a types package? -BW On Sun, Aug 10, 2014 at 9:36 PM, Chris Mattmann mattm...@apache.org wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24539/

Re: Review Request 24539: Fix Resource Manager in Trunk.

2014-08-18 Thread Chris Mattmann
Hi BW, what do you mean, do you mean is there a types package in the resource manager? Cheers, Chris -Original Message- From: BW bw...@mysoftcloud.com Reply-To: dev@oodt.apache.org Date: Monday, August 18, 2014 5:48 PM To: dev@oodt.apache.org, Chris Mattmann mattm...@apache.org

Re: Review Request 24539: Fix Resource Manager in Trunk.

2014-08-18 Thread BW
In general, have a preference to redefine types for common and widely used types. On Mon, Aug 18, 2014 at 5:54 PM, Chris Mattmann mattm...@apache.org wrote: Hi BW, what do you mean, do you mean is there a types package in the resource manager? Cheers, Chris -Original Message-

Re: Review Request 24539: Fix Resource Manager in Trunk.

2014-08-18 Thread Mattmann, Chris A (3980)
Hi BW, got it. Yes there is a structs package in Resource Manager. The File Manager, Workflow Manager, Resource Manager, Crawler, and Push Pull and PGE all follow this convention. This is where common component specific types live. Cheers, Chris

OODT Release Procedure Documentation

2014-08-18 Thread Lewis John Mcgibbney
Hi Folks, Is there are release procedure that we can view? If so then i can determine whether OODT-677 can be addressed, resolved, etc. Thanks Lewis -- *Lewis*

Re: OODT Release Procedure Documentation

2014-08-18 Thread Mattmann, Chris A (3980)
Hey bud https://cwiki.apache.org/confluence/display/OODT/Release+Process Sent from my iPhone On Aug 18, 2014, at 6:46 PM, Lewis John Mcgibbney lewis.mcgibb...@gmail.commailto:lewis.mcgibb...@gmail.com wrote: Hi Folks, Is there are release procedure that we can view? If so then i can

Re: OODT Release Procedure Documentation

2014-08-18 Thread Lewis John Mcgibbney
BOOM On Mon, Aug 18, 2014 at 6:56 PM, Mattmann, Chris A (3980) chris.a.mattm...@jpl.nasa.gov wrote: Hey bud https://cwiki.apache.org/confluence/display/OODT/Release+Process Sent from my iPhone On Aug 18, 2014, at 6:46 PM, Lewis John Mcgibbney

Accidental Commit to trunk

2014-08-18 Thread Lewis John Mcgibbney
Hi Folks, Apologies about the bothed commits to trunk. I forgot to add the -DdryRun=true argument to the mvn release:prepare execution. I've reverted the commit and deleted the release tag as follows lmcgibbn@LMC-032857 /usr/local/oodt(master) $ svn merge -r 1618784:1617659 . ...

Jenkins build became unstable: oodt-trunk ยป CAS Protocol IMAPS Implementation #1452

2014-08-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/oodt-trunk/org.apache.oodt$cas-protocol-imaps/1452/changes

Jenkins build is unstable: oodt-trunk #1452

2014-08-18 Thread Apache Jenkins Server
See https://builds.apache.org/job/oodt-trunk/1452/changes