Etienne, That is great to hear!
I am going to review and answer your prior emails but glad you got this going. Please feel free to contribute any updates to the wiki that you think make sense: https://cwiki.apache.org/confluence/display/OODT/Home We would definitely welcome them! Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: Etienne Koen <[email protected]> Date: Thursday, August 14, 2014 12:38 AM To: Chris Mattmann <[email protected]>, Thomas Bennett <[email protected]> Cc: "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Subject: RE: Remote data transfer >Hi Chris, > >Thanks for your help! No problem, please feel free to make our >conversation available to the community! > >I had success with ingesting a file to a remote server. Here is the >scenario I had: > >A file manager running 192.168.0.10 and a client, running on >192.168.0.11, wanting to ingest a file and archive it on the file manager >node (192.168.0.10). I used the tutorial at >https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guide >(Thanks Tom!) as an outline to configure or the necessary parameters, >e.g. the archive repository. > >Here are a list of variables I declared: > >export PROD_NAME=blah.txt >export PROD_REF=/root/source/blah.txt >export PROD_MET=/root/source/blah.txt.met > >using it with the command: > >./filemgr-client --url http://192.168.0.10:9000 --clientTransfer >--dataTransfer >org.apache.oodt.cas.filemgr.datatransfer.RemoteDataTransferFactory >--operation --ingestProduct --productName $PROD_NAME --productStructure >Flat --productTypeName GenericFile --metadataFile file://$PROD_MET --refs >file://$PROD_REF > >This gave a notification on the client side (192.168.0.11) that the file >was ingested successfully. The file appeared on the file manager node >(192.168.0.10) under the directory I specified :-) > >However, on the file manager node, I got a warning: > >WARNING: No Metadata specified for product [blah.txt] for required field >[DataVersion]: Attempting to continue processing metadata > >I guess this is due to not having a metadata extractor running? > >I am a bit of a newbie when it comes to cluster software, security etc.. >So here is what I have done prior to running the file manager to make >sure the environment was configured to allow communication: > >- I flushed the firewall settings and added a rule to allow communication > >$ iptables -F >$ iptables -I INPUT -j ACCEPT > >I then checked to see if all ports allowed communication: > >$ iptables -L > >Cheers >Etienne >________________________________________ >From: Mattmann, Chris A (3980) [[email protected]] >Sent: Wednesday, August 13, 2014 6:04 PM >To: Etienne Koen; Thomas Bennett >Cc: [email protected]; [email protected]; Mattmann, Chris A (3980) >Subject: Re: Remote data transfer > >Thanks guys. > >Etienne, I hope you don't mind but I've copied [email protected] > >on this email. That way you can tap into the entire Apache OODT >community for help. > >The URI has authority component is usually an error indicating >that you have referenced some environment variable in your config >(e.g., filemgr.properties in the etc directory) but that variable >isn't defined. E.g., maybe you have a *.policy.dirs property set >to file://[SOME_UNDEFINED_VARIABLE]/path/dir/ and SOME_UNDEFINED_VARIABLE >is undefined. > >Can you check that to see if that's the root cause of this issue? > >Cheers, >Chris > >------------------------ >Chris Mattmann >[email protected] > > > > >-----Original Message----- >From: Etienne Koen <[email protected]> >Date: Wednesday, August 13, 2014 1:42 AM >To: Thomas Bennett <[email protected]> >Cc: "[email protected]" <[email protected]>, Chris Mattmann ><[email protected]> >Subject: RE: Remote data transfer > >>Hi Tom, >> >>I get the following error when using the argument: >> >>ERROR: Failed to ingest product 'blah.txt' : URI has an authority >>component >> >>Here both the server and client were using port 9000 >> >>I get this when both the server and client are running on the same port >> >>When communicating on different ports I get: >> >><-- some I/O / HTTP exceptions --> >>... >>... >> >>ERROR: Failed to ingest product 'blah.txt' : Connection refused >> >>Server:9000 and Client:431 >> >>Do you know what any of this mean? >> >>Cheers >>Etienne >> >>________________________________________ >>From: Thomas Bennett [[email protected]] >>Sent: Wednesday, August 13, 2014 10:02 AM >>To: Etienne Koen >>Cc: [email protected]; [email protected] >>Subject: Re: Remote data transfer >> >>Hey Etienne, >> >>I've been out of the office the last week but I'm back now. >> >>./filemgr-client --url http://localhost:9000 --operation --ingestProduct >>--productName blah.txt --productStructure Flat --productTypeName >>GenericFile --metadataFile file:///tmp/blah.txt.met --refs >>file:///tmp/blah.txt >> >>How would this line be modified to achieve what I want to do? I see there >>is also an argument --clientTransfer --dataTransfer but I am not sure >>what java class to use for this? >> >>You will need to specify the filemgr remotely ie: --url >>http://192.168.0.1 - are you doing this? >> >>I've done remote file transfer before I'll see if I can remember how to >>do it. >> >>Can I log into the CHPC with the usual credentials? >> >>Cheers, >>Tom >>-- >>Thomas Bennett >> >>SKA South Africa >>Science Processing Team >> >>Office: +27 21 5067341 >>Mobile: +27 79 5237105 >> >>________________________________ >>Disclaimer: This E-mail message, including any attachments, is intended >>only for the person or entity to which it is addressed, and may contain >>confidential information. Each page attached hereto must also be read in >>conjunction with this disclaimer. >>If you are not the intended recipient you are hereby notified that any >>disclosure, copying, distribution or reliance upon the contents of this >>e-mail is strictly prohibited. E.&O.E. >> >>Disclaimer: This E-mail message, including any attachments, is intended >>only for the person or entity to which it is addressed, and may contain >>confidential information. Each page attached hereto must also be read in >>conjunction with this disclaimer. >>If you are not the intended recipient you are hereby notified that any >>disclosure, copying, distribution or reliance upon the contents of this >>e-mail is strictly prohibited. E.&O.E. > > >Disclaimer: This E-mail message, including any attachments, is intended >only for the person or entity to which it is addressed, and may contain >confidential information. Each page attached hereto must also be read in >conjunction with this disclaimer. >If you are not the intended recipient you are hereby notified that any >disclosure, copying, distribution or reliance upon the contents of this >e-mail is strictly prohibited. E.&O.E. > >Disclaimer: This E-mail message, including any attachments, is intended >only for the person or entity to which it is addressed, and may contain >confidential information. Each page attached hereto must also be read in >conjunction with this disclaimer. >If you are not the intended recipient you are hereby notified that any >disclosure, copying, distribution or reliance upon the contents of this >e-mail is strictly prohibited. E.&O.E.
