Re: PGE Question

[Cayanan, Michael D (388J)]

Hi Sheryl,


So, I took your suggestion and put this piece of code together in a post 
success ingest crawler action:


    String key = "L0b_Radar_Product_Filename";

    LOG.log(Level.INFO, "Workflow metadata keys: " + metadata.getAllKeys());

    Metadata productNameMet = new Metadata();

    productNameMet.addMetadata(key, product.getName());

      try {

        XmlRpcWorkflowManagerClient wClient = new XmlRpcWorkflowManagerClient(

                new 
URL(metadata.getMetadata(CoreMetKeys.WORKFLOW_MANAGER_URL)));

        passFlag = 
wClient.updateMetadataForWorkflow(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID),
 productNameMet);

        if (passFlag) {

          LOG.log(Level.INFO, "Updated workflow id '" + 
metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID) + "' with metadata: " + 
productNameMet.getAllKeys());

          Metadata after = 
wClient.getWorkflowInstanceMetadata(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID));

          LOG.log(Level.INFO, "Metadata after update: " + after.getAllKeys());

I was hoping that this would update my workflow metadata, however it doesn't 
appear to be working. I'm seeing this in my logs:

INFO: Workflow metadata keys: [FileLocation, TemporalRange~RangeEndingTime, 
ProductPathGroup~FileVersionId, StaticAncillaryFileGroup~L0b_Radar_MetDefFile, 
TemporalRange~RangeBeginningDate, WorkflowManagerUrl, TaskId, Filename, 
InputFileGroup~InputFilePath, StaticAncillaryFileGroup~L0b_Radar_ProductConfig, 
TemporalRange~RangeBeginningTime, WorkflowInstId, 
TemporalRange~RangeEndingDate, ProductType, JobId, ProductName, 
StaticAncillaryFileGroup~L0b_Radar_InputConfig, L0a_Radar_Product_Filename, 
ProcessingNode]
Mar 2, 2012 3:28:21 PM 
gov.nasa.smap.spdm.crawl.action.AddProductNameToWorkflowMetAction performAction
INFO: Updated workflow id '6333dd86-64bf-11e1-98c2-afe386bb7e05' with metadata: 
[L0b_Radar_Product_Filename]
Mar 2, 2012 3:28:22 PM 
gov.nasa.smap.spdm.crawl.action.AddProductNameToWorkflowMetAction performAction
INFO: Metadata after update: [L0b_Radar_Product_Filename]

Note that even getting the metadata via the workflow client API produces an odd 
result. It's only returning the single metadata key that I wanted to add to the 
current metadata in the workflow. Not sure why that is. In any case, when I 
look at the metadata in the task that follows this PGE, I only see the metadata 
without the extra metadata key that I wished to add.

It would seem like I'm doing this correctly, however I'm unsure if I'm missing 
something or if this is a bug in the code.

-Mike

From: Sheryl John <shery...@gmail.com<mailto:shery...@gmail.com>>
Reply-To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" 
<user@oodt.apache.org<mailto:user@oodt.apache.org>>
Date: Fri, 2 Mar 2012 12:58:53 -0800
To: "user@oodt.apache.org<mailto:user@oodt.apache.org>" 
<user@oodt.apache.org<mailto:user@oodt.apache.org>>
Subject: Re: PGE Question

Hi Mike,

You can use any of the Workflow Core met keys inside your PGE.
So, if you want to use the WorkflowInstId, you would specify it like this: 
[WorkflowInstId] or [JobId] inside your PGE script.

But there could be other ways to update workflow metadata. You could also have 
this in you metFileWriterClass, but that will be before your file is ingested.

On Fri, Mar 2, 2012 at 12:36 PM, Cayanan, Michael D (388J) 
<michael.d.caya...@jpl.nasa.gov<mailto:michael.d.caya...@jpl.nasa.gov>> wrote:
Hi,

In my PGE, I specify my output like the following:


<output>

           <dir path="[ProductLocation]" createBeforeExe="true">

              <files regExp="SMAP_L0B_RADAR_SPS.*\.h5" 
metFileWriterClass="[MetFileWriter]" args="[ConfigMetoutFile]" />

              <files name="[PGETask_Name]_RunConfig_[ProductionDateTime].xml" 
metFileWriterClass="[MetFileWriter]" args="[RunConfigMetoutFile]" />

   </dir>

</output>


After ingestion of this output product, I'd like to save the filename in my 
workflow metadata. How would I do this?


I tried creating a PostIngestSuccess action, but found out that whatever 
metadata saving you want to do on the "crawler" side doesn't get passed back to 
the workflow metadata.


I suppose I could update the workflow metadata by creating a PostIngestSuccess 
action and using the workflow manager client API to do this. However, when I 
look at this class, I see the following method,


updateMetadataForWorkflow(String workflowInstId, Metadata metadata)


where it requires the workflow instance ID as a parameter. Is there a way to 
capture the workflow instance ID into the workflow metadata so that I can 
leverage this method to update my metadata?


Thanks,

Mike





--
-Sheryl