Just to follow up, I have reissued a new host certificate / container cert
on host A, with no effect. However, I'm sure the globus user certificate
(and likewise the simpleCA these are coming from) are OK, because I was able
to use the same globus user on a different host and jobs are working fine.
Host A is running a 4.1.0 container, for the record.
Thanks,
Adam
On 11/5/07, Adam Bazinet <[EMAIL PROTECTED]> wrote:
>
> Hi All,
>
> Our Grid, which has been functioning perfectly well, seems to have broken
> now all of a sudden in the sense that all GRAM job submissions from host A
> to host B, C, D, etc (where B C D are independent remote sites) are failing
> with the following error:
>
> 2007-11-05 16:54:38,950 INFO
> exec.StateMachine[RunQueueThread_0,logJobAccepted:3193] Job
> b347a4d0-8be9-11dc-a3ac-ba9ad65cba4d accepted for local user 'globus'
> 2007-11-05 16:54:39,457 ERROR
> exec.StateMachine[RunQueueThread_0,fileCleanUp:2730] A secondary fault
> occured while trying
> to gracefully fail.
> AxisFault
> faultCode:
> {http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}General<http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd%7DGeneral>
> faultSubcode:
> faultString: [Caused by: Conversion failed [Caused by:
> java.io.IOException: Not a UTC_TIME]]
> faultActor:
> faultNode:
> faultDetail:
>
> {http://xml.apache.org/axis/}stackTrace<http://xml.apache.org/axis/%7DstackTrace>:.
> Caused by Conversion failed. Caused by
> COM.claymoresystems.cert.CertificateDecodeException : java.io.IOException:
> Not a UTC_TIME
> at COM.claymoresystems.cert.X509Cert.&lt;init&gt;(
> X509Cert.java:209)
> at org.globus.gsi.ptls.PureTLSUtil.certificateChainToVector(
> PureTLSUtil.java :226)
> at org.globus.gsi.proxy.ProxyPathValidator.validate(
> ProxyPathValidator.java:216)
> at org.globus
> .wsrf.impl.security.authentication.wssec.WSSecurityEngine.verifyXMLSignature(
> WSSecurityEngine.java:299)
> at org.globus
> .wsrf.impl.security.authentication.wssec.WSSecurityRequestEngine.verifyXMLSignature(
> WSSecurityRequestEngine.java:97)
> at
> org.globus.wsrf.impl.security.authentication.wssec.WSSecurityEngine.handleSignatureElement
> (WSSecurityEngine.java:116)
> at org.globus
> .wsrf.impl.security.authentication.wssec.WSSecurityEngine.processSecurityHeader(
> WSSecurityEngine.java:516)
> at
> org.globus.wsrf.impl.security.authentication.wssec.WSSecurityEngine.processSecurityHeader
> (WSSecurityEngine.java:482)
> at org.globus
> .wsrf.impl.security.authentication.wssec.WSSecurityEngine.processSecurityHeader(
> WSSecurityEngine.java:397)
> at
> org.globus.wsrf.impl.security.authentication.wssec.WSSecurityRequestEngine.processSecurityHeader
> (WSSecurityRequestEngine.java:61)
> at org.globus
> .wsrf.impl.security.authentication.wssec.WSSecurityBasicHandler.handleMessage(
> WSSecurityBasicHandler.java:43)
> at
> org.globus.wsrf.impl.security.authentication.wssec.WSSecurityHandler.handleRequest
> (WSSecurityHandler.java:21)
> at org.apache.axis.handlers.HandlerChainImpl.handleRequest(
> HandlerChainImpl.java:105)
> at org.apache.axis.handlers.JAXRPCHandler.invoke(
> JAXRPCHandler.java:52)
> at org.globus.wsrf.handlers.JAXRPCHandler.invoke(
> JAXRPCHandler.java:26)
> at org.apache.axis.strategies.InvocationStrategy.visit(
> InvocationStrategy.java:32)
> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java :118)
> at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
> at org.apache.axis.server.AxisServer.invoke(AxisServer.java:248)
> at org.apache.axis.transport.local.LocalSender.invoke(
> LocalSender.java :141)
> at org.apache.axis.strategies.InvocationStrategy.visit(
> InvocationStrategy.java:32)
> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
> at org.apache.axis.SimpleChain.invoke (SimpleChain.java:83)
> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
> at org.apache.axis.client.Call.invokeEngine(Call.java:2727)
> at org.apache.axis.client.Call.invoke(Call.java :2710)
> at org.apache.axis.client.Call.invoke(Call.java:2386)
> at org.apache.axis.client.Call.invoke(Call.java:2309)
> at org.apache.axis.client.Call.invoke(Call.java:1766)
> at
> org.globus.rft.generated.bindings.ReliableFileTransferFactoryPortTypeSOAPBindingStub.createReliableFileTransfer
> (ReliableFileTransferFactoryPortTypeSOAPBindingStub.java:874)
> at org.globus
> .exec.service.exec.utils.StagingHelper.submitStagingRequest(
> StagingHelper.java:168)
> at org.globus.exec.service.exec.StateMachine.fileCleanUp (
> StateMachine.java:2716)
> at org.globus
> .exec.service.exec.StateMachine.processFailureFileCleanUpState(
> StateMachine.java:2091)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke (Method.java:585)
> at org.globus.exec.service.exec.StateMachine.processState(
> StateMachine.java:302)
> at org.globus.exec.service.exec.RunThread.run(RunThread.java:85)
>
> {
> http://xml.apache.org/axis/}hostname:seil.umd.edu<http://xml.apache.org/axis/%7Dhostname:seil.umd.edu>
>
> [Caused by: Conversion failed [Caused by: java.io.IOException: Not a
> UTC_TIME]]
> at org.apache.axis.message.SOAPFaultBuilder.createFault(
> SOAPFaultBuilder.java :221)
> at org.apache.axis.message.SOAPFaultBuilder.endElement(
> SOAPFaultBuilder.java:128)
> at org.apache.axis.encoding.DeserializationContext.endElement(
> DeserializationContext.java:1087)
> at org.apache.xerces.parsers.AbstractSAXParser.endElement (Unknown
> Source)
> at
> org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown
> Source)
> at
> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
> Source)
> at
> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
> Source)
> at org.apache.xerces.parsers.XML11Configuration.parse(Unknown
> Source)
> at org.apache.xerces.parsers.XML11Configuration.parse (Unknown
> Source)
> at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
> at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown
> Source)
> at javax.xml.parsers.SAXParser.parse(SAXParser.java :375)
> at org.apache.axis.encoding.DeserializationContext.parse(
> DeserializationContext.java:227)
> at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:645)
> at org.apache.axis.Message.getSOAPEnvelope (Message.java:424)
> at
> org.apache.axis.message.addressing.handler.AddressingHandler.processClientResponse
> (AddressingHandler.java:305)
> at
> org.apache.axis.message.addressing.handler.AddressingHandler.invoke (
> AddressingHandler.java:110)
> at org.apache.axis.strategies.InvocationStrategy.visit(
> InvocationStrategy.java:32)
> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
> at org.apache.axis.SimpleChain.invoke (SimpleChain.java:83)
> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:190)
> at org.apache.axis.client.Call.invokeEngine(Call.java:2727)
> at org.apache.axis.client.Call.invoke(Call.java :2710)
> at org.apache.axis.client.Call.invoke(Call.java:2386)
> at org.apache.axis.client.Call.invoke(Call.java:2309)
> at org.apache.axis.client.Call.invoke(Call.java:1766)
> at
> org.globus.rft.generated.bindings.ReliableFileTransferFactoryPortTypeSOAPBindingStub.createReliableFileTransfer
> (ReliableFileTransferFactoryPortTypeSOAPBindingStub.java:874)
> at org.globus
> .exec.service.exec.utils.StagingHelper.submitStagingRequest(
> StagingHelper.java:168)
> at org.globus.exec.service.exec.StateMachine.fileCleanUp (
> StateMachine.java:2716)
> at org.globus
> .exec.service.exec.StateMachine.processFailureFileCleanUpState(
> StateMachine.java:2091)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke (Method.java:585)
> at org.globus.exec.service.exec.StateMachine.processState(
> StateMachine.java:302)
> at org.globus.exec.service.exec.RunThread.run(RunThread.java:85)
> 2007-11-05 16:54:40,046 INFO
> exec.StateMachine[RunQueueThread_0,logJobFailed:3212] Job
> b347a4d0-8be9-11dc-a3ac-ba9ad65cba4d failed
>
> What is this "Not a UTC_TIME" stuff? I've checked and re-checked the
> certs, they are all valid and I can't think of anything that has changed,
> except daylight savings! If anyone has ideas, please let me know ASAP!
>
> Thanks,
> Adam
>
