Hi Chris, Thanks for your help! No problem, please feel free to make our conversation available to the community!
I had success with ingesting a file to a remote server. Here is the scenario I had: A file manager running 192.168.0.10 and a client, running on 192.168.0.11, wanting to ingest a file and archive it on the file manager node (192.168.0.10). I used the tutorial at https://cwiki.apache.org/confluence/display/OODT/OODT+Filemgr+User+Guide (Thanks Tom!) as an outline to configure or the necessary parameters, e.g. the archive repository. Here are a list of variables I declared: export PROD_NAME=blah.txt export PROD_REF=/root/source/blah.txt export PROD_MET=/root/source/blah.txt.met using it with the command: ./filemgr-client --url http://192.168.0.10:9000 --clientTransfer --dataTransfer org.apache.oodt.cas.filemgr.datatransfer.RemoteDataTransferFactory --operation --ingestProduct --productName $PROD_NAME --productStructure Flat --productTypeName GenericFile --metadataFile file://$PROD_MET --refs file://$PROD_REF This gave a notification on the client side (192.168.0.11) that the file was ingested successfully. The file appeared on the file manager node (192.168.0.10) under the directory I specified :-) However, on the file manager node, I got a warning: WARNING: No Metadata specified for product [blah.txt] for required field [DataVersion]: Attempting to continue processing metadata I guess this is due to not having a metadata extractor running? I am a bit of a newbie when it comes to cluster software, security etc.. So here is what I have done prior to running the file manager to make sure the environment was configured to allow communication: - I flushed the firewall settings and added a rule to allow communication $ iptables -F $ iptables -I INPUT -j ACCEPT I then checked to see if all ports allowed communication: $ iptables -L Cheers Etienne ________________________________________ From: Mattmann, Chris A (3980) [[email protected]] Sent: Wednesday, August 13, 2014 6:04 PM To: Etienne Koen; Thomas Bennett Cc: [email protected]; [email protected]; Mattmann, Chris A (3980) Subject: Re: Remote data transfer Thanks guys. Etienne, I hope you don't mind but I've copied [email protected] on this email. That way you can tap into the entire Apache OODT community for help. The URI has authority component is usually an error indicating that you have referenced some environment variable in your config (e.g., filemgr.properties in the etc directory) but that variable isn't defined. E.g., maybe you have a *.policy.dirs property set to file://[SOME_UNDEFINED_VARIABLE]/path/dir/ and SOME_UNDEFINED_VARIABLE is undefined. Can you check that to see if that's the root cause of this issue? Cheers, Chris ------------------------ Chris Mattmann [email protected] -----Original Message----- From: Etienne Koen <[email protected]> Date: Wednesday, August 13, 2014 1:42 AM To: Thomas Bennett <[email protected]> Cc: "[email protected]" <[email protected]>, Chris Mattmann <[email protected]> Subject: RE: Remote data transfer >Hi Tom, > >I get the following error when using the argument: > >ERROR: Failed to ingest product 'blah.txt' : URI has an authority >component > >Here both the server and client were using port 9000 > >I get this when both the server and client are running on the same port > >When communicating on different ports I get: > ><-- some I/O / HTTP exceptions --> >... >... > >ERROR: Failed to ingest product 'blah.txt' : Connection refused > >Server:9000 and Client:431 > >Do you know what any of this mean? > >Cheers >Etienne > >________________________________________ >From: Thomas Bennett [[email protected]] >Sent: Wednesday, August 13, 2014 10:02 AM >To: Etienne Koen >Cc: [email protected]; [email protected] >Subject: Re: Remote data transfer > >Hey Etienne, > >I've been out of the office the last week but I'm back now. > >./filemgr-client --url http://localhost:9000 --operation --ingestProduct >--productName blah.txt --productStructure Flat --productTypeName >GenericFile --metadataFile file:///tmp/blah.txt.met --refs >file:///tmp/blah.txt > >How would this line be modified to achieve what I want to do? I see there >is also an argument --clientTransfer --dataTransfer but I am not sure >what java class to use for this? > >You will need to specify the filemgr remotely ie: --url >http://192.168.0.1 - are you doing this? > >I've done remote file transfer before I'll see if I can remember how to >do it. > >Can I log into the CHPC with the usual credentials? > >Cheers, >Tom >-- >Thomas Bennett > >SKA South Africa >Science Processing Team > >Office: +27 21 5067341 >Mobile: +27 79 5237105 > >________________________________ >Disclaimer: This E-mail message, including any attachments, is intended >only for the person or entity to which it is addressed, and may contain >confidential information. Each page attached hereto must also be read in >conjunction with this disclaimer. >If you are not the intended recipient you are hereby notified that any >disclosure, copying, distribution or reliance upon the contents of this >e-mail is strictly prohibited. E.&O.E. > >Disclaimer: This E-mail message, including any attachments, is intended >only for the person or entity to which it is addressed, and may contain >confidential information. Each page attached hereto must also be read in >conjunction with this disclaimer. >If you are not the intended recipient you are hereby notified that any >disclosure, copying, distribution or reliance upon the contents of this >e-mail is strictly prohibited. E.&O.E. Disclaimer: This E-mail message, including any attachments, is intended only for the person or entity to which it is addressed, and may contain confidential information. Each page attached hereto must also be read in conjunction with this disclaimer. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or reliance upon the contents of this e-mail is strictly prohibited. E.&O.E. Disclaimer: This E-mail message, including any attachments, is intended only for the person or entity to which it is addressed, and may contain confidential information. Each page attached hereto must also be read in conjunction with this disclaimer. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or reliance upon the contents of this e-mail is strictly prohibited. E.&O.E.
