Hi Mike,
On Mar 2, 2012, at 3:43 PM, Cayanan, Michael D (388J) wrote:
> Hi Sheryl,
>
> So, I took your suggestion and put this piece of code together in a post
> success ingest crawler action:
>
> String key = "L0b_Radar_Product_Filename";
> LOG.log(Level.INFO, "Workflow metadata keys: " + metadata.getAllKeys());
>
> Metadata productNameMet = new Metadata();
> productNameMet.addMetadata(key, product.getName());
> try {
> XmlRpcWorkflowManagerClient wClient = new XmlRpcWorkflowManagerClient(
> new
> URL(metadata.getMetadata(CoreMetKeys.WORKFLOW_MANAGER_URL)));
> passFlag =
> wClient.updateMetadataForWorkflow(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID),
> productNameMet);
> if (passFlag) {
> LOG.log(Level.INFO, "Updated workflow id '" +
> metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID) + "' with metadata: " +
> productNameMet.getAllKeys());
> Metadata after =
> wClient.getWorkflowInstanceMetadata(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID));
> LOG.log(Level.INFO, "Metadata after update: " + after.getAllKeys());
>
>
>
> I was hoping that this would update my workflow metadata, however it doesn't
> appear to be working. I'm seeing this in my logs:
>
> INFO: Workflow metadata keys: [FileLocation, TemporalRange~RangeEndingTime,
> ProductPathGroup~FileVersionId,
> StaticAncillaryFileGroup~L0b_Radar_MetDefFile,
> TemporalRange~RangeBeginningDate, WorkflowManagerUrl, TaskId, Filename,
> InputFileGroup~InputFilePath,
> StaticAncillaryFileGroup~L0b_Radar_ProductConfig,
> TemporalRange~RangeBeginningTime, WorkflowInstId,
> TemporalRange~RangeEndingDate, ProductType, JobId, ProductName,
> StaticAncillaryFileGroup~L0b_Radar_InputConfig, L0a_Radar_Product_Filename,
> ProcessingNode]
> Mar 2, 2012 3:28:21 PM
> gov.nasa.smap.spdm.crawl.action.AddProductNameToWorkflowMetAction
> performAction
> INFO: Updated workflow id '6333dd86-64bf-11e1-98c2-afe386bb7e05' with
> metadata: [L0b_Radar_Product_Filename]
> Mar 2, 2012 3:28:22 PM
> gov.nasa.smap.spdm.crawl.action.AddProductNameToWorkflowMetAction
> performAction
> INFO: Metadata after update: [L0b_Radar_Product_Filename]
>
> Note that even getting the metadata via the workflow client API produces an
> odd result.
Not sure why this is odd.
In line: String key = "L0b_Radar_Product_Filename";
You set the key name to: L0b_Radar_Product_Filename
Then, in line: Metadata productNameMet = new Metadata();
You create a new met object.
Then in line: productNameMet.addMetadata(key, product.getName());
You add a single key to that new met object with the product value.
Then in line: passFlag =
wClient.updateMetadataForWorkflow(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID),
productNameMet);
You update the workflow instance with the ID, passing it the productNameMet,
thus replacing the metadata for this workflow instance, with
the one with only a single key.
Then in line: Metadata after =
wClient.getWorkflowInstanceMetadata(metadata.getMetadata(CoreMetKeys.WORKFLOW_INST_ID));
You get the metadata for the workflow which should now only have 1 key.
And then in line: LOG.log(Level.INFO, "Metadata after update: " +
after.getAllKeys());
You print the key names and you see 1 key name.
> It's only returning the single metadata key that I wanted to add to the
> current metadata in the workflow.
To do that you need to merge your new product met key + val with the existing
workflow met. Try doing a get Workflow Instance metadata first, then
appending your key to it, then saving that back.
> Not sure why that is. In any case, when I look at the metadata in the task
> that follows this PGE, I only see the metadata without the extra metadata key
> that I wished to add.
>
> It would seem like I'm doing this correctly, however I'm unsure if I'm
> missing something or if this is a bug in the code.
See above, Mike. It's operating as expected but you just need to update it to
plumb through the original met too.
HTH!
Cheers,
Chris
>
> -Mike
>
> From: Sheryl John <[email protected]>
> Reply-To: "[email protected]" <[email protected]>
> Date: Fri, 2 Mar 2012 12:58:53 -0800
> To: "[email protected]" <[email protected]>
> Subject: Re: PGE Question
>
> Hi Mike,
>
> You can use any of the Workflow Core met keys inside your PGE.
> So, if you want to use the WorkflowInstId, you would specify it like this:
> [WorkflowInstId] or [JobId] inside your PGE script.
>
> But there could be other ways to update workflow metadata. You could also
> have this in you metFileWriterClass, but that will be before your file is
> ingested.
>
> On Fri, Mar 2, 2012 at 12:36 PM, Cayanan, Michael D (388J)
> <[email protected]> wrote:
>> Hi,
>>
>> In my PGE, I specify my output like the following:
>>
>> <output>
>> <dir path="[ProductLocation]" createBeforeExe="true">
>> <files regExp="SMAP_L0B_RADAR_SPS.*\.h5"
>> metFileWriterClass="[MetFileWriter]" args="[ConfigMetoutFile]" />
>> <files
>> name="[PGETask_Name]_RunConfig_[ProductionDateTime].xml"
>> metFileWriterClass="[MetFileWriter]" args="[RunConfigMetoutFile]" />
>> </dir>
>> </output>
>>
>> After ingestion of this output product, I'd like to save the filename in my
>> workflow metadata. How would I do this?
>>
>> I tried creating a PostIngestSuccess action, but found out that whatever
>> metadata saving you want to do on the "crawler" side doesn't get passed back
>> to the workflow metadata.
>>
>> I suppose I could update the workflow metadata by creating a
>> PostIngestSuccess action and using the workflow manager client API to do
>> this. However, when I look at this class, I see the following method,
>>
>> updateMetadataForWorkflow(String workflowInstId, Metadata metadata)
>>
>>
>> where it requires the workflow instance ID as a parameter. Is there a way to
>> capture the workflow instance ID into the workflow metadata so that I can
>> leverage this method to update my metadata?
>>
>> Thanks,
>> Mike
>>
>>
>
>
>
> --
> -Sheryl
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [email protected]
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
