John,
Change:
-classpath "$FILEMGR_HOME"/lib \
To:
-classpath "$FILEMGR_HOME/lib/*" \
-Michael
On Thu, Mar 12, 2015 at 1:37 PM, John Reynolds <[email protected]> wrote:
> Thanks Michael, if i modify the filemgr-client to look like this (at the
> end)
> "$_RUNJAVA" $JAVA_OPTS $OODT_OPTS \
> -classpath "$FILEMGR_HOME"/lib \
>
> -Dorg.apache.oodt.cas.filemgr.properties="$FILEMGR_HOME"/etc/filemgr.properties
> \
> -Djava.util.logging.config.file="$FILEMGR_HOME"/etc/logging.properties \
>
> -Dorg.apache.oodt.cas.cli.action.spring.config=file:"$FILEMGR_HOME"/policy/cmd-line-actions.xml
> \
>
> -Dorg.apache.oodt.cas.cli.option.spring.config=file:"$FILEMGR_HOME"/policy/cmd-line-options.xml
> \
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient "$@“
>
> (replacing ext jars with -classpath) then i get
> Error: Could not find or load main class
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient
> i assume i’m doing something wrong with classpath but not sure what
>
> > On Mar 12, 2015, at 11:31 AM, Michael Starch <[email protected]> wrote:
> >
> > John,
> >
> > Can you open filemgr-client sh script? It may set the JAVA_EXT_JARS
> > there. If so, it is clobbering the default path for "extension" jars,
> and
> > your java encryption jars are not being picked up. If it does set
> > JAVA_EXT_JARS you have two options:
> >
> > 1. Move all your encryption jars into FILEMGR_HOME/lib/
> > 2. update filemgr-client script to us classpath to specify the jars in
> the
> > FILEMGRHOME/lib directory and remove the use of JAVA_EXT_JARS
> >
> >
> > -Michael
> >
> >
> > On Thu, Mar 12, 2015 at 11:12 AM, John Reynolds <[email protected]>
> wrote:
> >
> >> Hi Michael
> >> yeah it’s openjdk 1.7 (“1.7.0_75")
> >> i did download the the unlimited encryption jar from oracle and replaced
> >> the local_policy / us_export_policy jars in javahome/jre/lib/security
> >> more i read, maybe limited by jce.jar
> >>
> >> i dont have anything special set for extension jars
> >>
> >>
> >>
> >>> On Mar 12, 2015, at 10:35 AM, Michael Starch <[email protected]>
> wrote:
> >>>
> >>> John,
> >>>
> >>> What version of the JDK are you running, and what is your extension
> jars
> >>> environment variable set to. Do you have the java cryptology jar
> >> included
> >>> (Oracle JDK usually has this, I don't know if Open JDK does).
> >>>
> >>> "Algorithm HmacSHA1 not available" is usually thrown when Java cannot
> >> find
> >>> the java crypto jar used to calculate the given hash.
> >>>
> >>> -Michael
> >>>
> >>> On Thu, Mar 12, 2015 at 9:06 AM, John Reynolds <[email protected]>
> >> wrote:
> >>>
> >>>> Hi Lewis,
> >>>> using the latest docker buggtb/oodt image, which i assume is .8
> >>>> here’s the command i’m running to test the upload
> >>>>
> >>>> filemgr-client --url http://localhost:9000 --operation
> --ingestProduct
> >>>> --productName test --productStructure Flat --productTypeName
> GenericFile
> >>>> --metadataFile file:///root/test.txt.met --refs file:///root/test.txt
> >>>>
> >>>> i verified that i can upload to the path using the s3 tools on the
> box /
> >>>> with same credentials i put in the properties file
> >>>>
> >>>> here’s the full exception returned:
> >>>>
> >>>> rg.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException:
> >>>> org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException:
> >>>> Failed to upload product reference /root/test.txt to S3 at
> >>>> usr/src/oodt/data/archive/test/test.txt
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:768)
> >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>>> at
> >>>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>>> at
> >>>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>> at java.lang.reflect.Method.invoke(Method.java:606)
> >>>> at org.apache.xmlrpc.Invoker.execute(Invoker.java:130)
> >>>> at
> >>>> org.apache.xmlrpc.XmlRpcWorker.invokeHandler(XmlRpcWorker.java:84)
> >>>> at org.apache.xmlrpc.XmlRpcWorker.execute(XmlRpcWorker.java:146)
> >>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:139)
> >>>> at org.apache.xmlrpc.XmlRpcServer.execute(XmlRpcServer.java:125)
> >>>> at
> org.apache.xmlrpc.WebServer$Connection.run(WebServer.java:761)
> >>>> at org.apache.xmlrpc.WebServer$Runner.run(WebServer.java:642)
> >>>> at java.lang.Thread.run(Thread.java:745)
> >>>> Caused by:
> >>>> org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException:
> >>>> Failed to upload product reference /root/test.txt to S3 at
> >>>> usr/src/oodt/data/archive/test/test.txt
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.datatransfer.S3DataTransferer.transferProduct(S3DataTransferer.java:78)
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManager.ingestProduct(XmlRpcFileManager.java:752)
> >>>> ... 12 more
> >>>> Caused by: com.amazonaws.AmazonClientException: Unable to calculate a
> >>>> request signature: Unable to calculate a request signature: Algorithm
> >>>> HmacSHA1 not available
> >>>> at
> >>>>
> >>
> com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:71)
> >>>> at
> >>>>
> >>
> com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:57)
> >>>> at
> >>>> com.amazonaws.services.s3.internal.S3Signer.sign(S3Signer.java:128)
> >>>> at
> >>>>
> >>
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:330)
> >>>> at
> >>>> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:232)
> >>>> at
> >>>>
> >>
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3528)
> >>>> at
> >>>>
> >>
> com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1393)
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.datatransfer.S3DataTransferer.transferProduct(S3DataTransferer.java:76)
> >>>> ... 13 more
> >>>> Caused by: com.amazonaws.AmazonClientException: Unable to calculate a
> >>>> request signature: Algorithm HmacSHA1 not available
> >>>> at
> >>>> com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:90)
> >>>> at
> >>>>
> >>
> com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:68)
> >>>> ... 20 more
> >>>> Caused by: java.security.NoSuchAlgorithmException: Algorithm HmacSHA1
> >> not
> >>>> available
> >>>> at javax.crypto.Mac.getInstance(Mac.java:176)
> >>>> at
> >>>> com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:86)
> >>>> ... 21 more
> >>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception:
> >>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
> >>>> ingesting product
> [org.apache.oodt.cas.filemgr.structs.Product@6454bbe1
> >> ]
> >>>> :
> org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException:
> >>>> Failed to upload product reference /root/test.txt to S3 at
> >>>> usr/src/oodt/data/archive/test/test.txt
> >>>> at
> >>>>
> >>
> org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcClientResponseProcessor.java:104)
> >>>> at
> >>>>
> >>
> org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcClientResponseProcessor.java:71)
> >>>> at
> >>>>
> org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73)
> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194)
> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185)
> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178)
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1198)
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:112)
> >>>> at
> >>>>
> org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
> >>>> at
> >>>> org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1350)
> >>>> java.lang.Exception: Failed to ingest product
> >>>> [org.apache.oodt.cas.filemgr.structs.Product@7f25ab41] :
> >>>> java.lang.Exception:
> >>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
> >>>> ingesting product
> [org.apache.oodt.cas.filemgr.structs.Product@6454bbe1
> >> ]
> >>>> :
> org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException:
> >>>> Failed to upload product reference /root/test.txt to S3 at
> >>>> usr/src/oodt/data/archive/test/test.txt
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.ingestProduct(XmlRpcFileManagerClient.java:1303)
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.cli.action.IngestProductCliAction.execute(IngestProductCliAction.java:112)
> >>>> at
> >>>>
> org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331)
> >>>> at
> >>>> org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187)
> >>>> at
> >>>>
> >>
> org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFileManagerClient.java:1350)
> >>>> ERROR: Failed to ingest product 'test' : Failed to ingest product
> >>>> [org.apache.oodt.cas.filemgr.structs.Product@7f25ab41] :
> >>>> java.lang.Exception:
> >>>> org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Error
> >>>> ingesting product
> [org.apache.oodt.cas.filemgr.structs.Product@6454bbe1
> >> ]
> >>>> :
> org.apache.oodt.cas.filemgr.structs.exceptions.DataTransferException:
> >>>> Failed to upload product reference /root/test.txt to S3 at
> >>>> usr/src/oodt/data/archive/test/test.txt
> >>>>
> >>>>
> >>>>> On Mar 11, 2015, at 6:21 PM, Lewis John Mcgibbney <
> >>>> [email protected]> wrote:
> >>>>>
> >>>>> Hi John,
> >>>>> What beau on of OODT are you using?
> >>>>> The file types should be encapsulated by the factory implementation I
> >>>> would
> >>>>> guess. It should be. Case of mike this the default filemgr store then
> >> off
> >>>>> you go.
> >>>>> Can you provide a paste of how your arriving at your exception
> please?
> >>>>> Lewia
> >>>>>
> >>>>> On Wednesday, March 11, 2015, John Reynolds <[email protected]>
> >> wrote:
> >>>>>
> >>>>>> Should S3DataTransferer work with the generic file type from the get
> >> go
> >>>> or
> >>>>>> is there anything i need to change
> >>>>>> i was using the radix docker but getting 'Caused by:
> >>>>>> java.security.NoSuchAlgorithmException: Algorithm HmacSHA1 not
> >>>> available’
> >>>>>> i tried downloading the jce but no dice
> >>>>>> i might try just building on my local machine and trying again
> >>>>>>
> >>>>>> Thanks
> >>>>>>
> >>>>>>> On Mar 10, 2015, at 6:19 PM, Lewis John Mcgibbney <
> >>>>>> [email protected] <javascript:;>> wrote:
> >>>>>>>
> >>>>>>> Hey John,
> >>>>>>> You're right, you've nearly answered everything here ;)
> >>>>>>> OK, so your properties can be located here
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://github.com/apache/oodt/blob/trunk/filemgr/src/main/resources/filemgr.properties#L114-L124
> >>>>>>> You'll see the usual key, value pairs in there which you can ad
> >> before
> >>>>>>> build the project and packaging filemgr.properties along with your
> >>>>>> compiled
> >>>>>>> code.
> >>>>>>> Regarding the following,
> >>>>>>>
> >>>>>>> On Tue, Mar 10, 2015 at 4:31 PM, John Reynolds <
> [email protected]
> >>>>>> <javascript:;>> wrote:
> >>>>>>>
> >>>>>>>>
> >>>>>>>> in the near future, i will want to pass in a security token as
> well
> >> to
> >>>>>> the
> >>>>>>>> S3 client. to do so i would extend or modify the s3datatransfer
> >> class
> >>>>>>>> (which expires after an hour), what’s the best practice to read in
> >>>> this
> >>>>>>>> transient data from the command line (or wherever else i would
> >>>> initiate
> >>>>>> a
> >>>>>>>> job from) since this wouldnt go in a properties file
> >>>>>>>
> >>>>>>>
> >>>>>>> Do you mean that your security token expires hourly? As oppose to
> >> your
> >>>> s3
> >>>>>>> client?
> >>>>>>> Lets try and defined exactly what you are after here.
> >>>>>>> Thanks
> >>>>>>> Lewis
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> *Lewis*
> >>>>
> >>>>
> >>
> >>
>
>
