Hey Sheryl, Are you sure that RecordID is being recorded for the particular products that you are trying to look up?
One way to find this out: 1. get the product IDs back for the products you are interested in. One way to do this would be to remove RecordID from your query as you mention below and then add CAS.ProductId to the SELECT list, and to change FORMAT='$FileLocation...' to FORMAT='$CAS.ProductId'. 2. Then, for each of the product IDs, check out OODT-306 [1] and use the metdump alias to dump out the metadata for each product ID to the current working directory. Then, inspect that .met file. Is RecordID recorded? Let's start there. Cheers, Chris [1] http://issues.apache.org/jira/browse/OODT-306 On Sep 15, 2011, at 8:53 PM, Sheryl John wrote: > Hi, > > I am not clear about the metadata in .met files generated by the > met-config.xml files. Particularly, where do they fall in the product-type > and element mappings? > > Because in my query, SQL(FORMAT='$FileLocation/$Filename'){ SELECT > FileLocation,Filename,ISMTable,RecordID FROM ISMRawData WHERE ISMTable = > 'Chartevents' AND RecordID = '[PID]' , I'm selecting elements that are > mapped to the product-type, ISMRawData and elements that are mapped to the > GenericFile product-type. > > Except RecordID, which was defined as a key in the metout-config file and > I'm assuming that the parsing error is because the RecordID is not mapped to > any product-type. This query works and pulls all the files I need when I > remove RecordID from the query. Again, since I'm not clear about metadata > mapping , it's highly likely that I'm wrong about this. > > Thanks, > On Thu, Sep 15, 2011 at 11:17 AM, holenoter <[email protected]> wrote: > hey Sheryl, > > I think there is a misunderstanding on how the met-config.xml files are > used... the metadata in these files are not added to your workflow context > metadata... they are only used to create the *.met files for the data file > for filemgr ingest... about your query I'm not quite clear on how you are > wanting go use it... it kinda seems like you want to dynamically replace out > RecordIO... in which case you're gonna need it to look more like RecordIO = > '[PID]' > > -brian > > > On Sep 14, 2011, at 09:18 PM, Sheryl John <[email protected]> wrote: > >> Hi Chris, >> >> Thanks for pointing that out. I was working on a different query and similar >> to the one I described. Though I've used only single quotes now, I'm still >> getting the same parsing error. >> >> But, I think what I intended to do with context metadata ( for my workflow) >> doesn't make sense for a single file product and rather, would apply for a >> group of files. >> I guess I have revise my workflow logic and metadata use before getting back >> to this query. >> >> Thanks! >> >> >> >> On Wed, Sep 14, 2011 at 8:42 PM, Mattmann, Chris A (388J) >> <[email protected]> wrote: >> Hi Sheryl, >> >> I notice you're using double quotes on the RecordID = "PID", yet you are >> using single quotes on ISMTable = 'Chartevents'. >> >> Try using single quotes on both and see if that fixes it. >> >> Thanks! >> >> Cheers, >> Chris >> >> >> On Sep 14, 2011, at 7:03 PM, Sheryl John wrote: >> >> > Hi, >> > >> > I have defined some key-val pairs for a file( say Output.csv) in a >> > metout-config.xml for my PGETask Workflow. However, after executing the >> > workflow, the met-config.xml is not creating a Output.csv.cas file. >> > >> > I want to be able to use the above keys/metadata later on in an SQL-like >> > query from the pgeconfig file. For example, if I've defined 'RecordID' as >> > a key in the metout-config.xml, I would want to use this metadata in the >> > following query: >> > >> > SQL(FORMAT='$FileLocation/$Filename'){ SELECT >> > FileLocation,Filename,ISMTable,RecordID FROM ISMRawData WHERE ISMTable = >> > 'Chartevents' AND RecordID = "PID"} The others keys included in the >> > query above are elements and product-types that were defined during >> > ingestion in the File Manager. >> > >> > At present, the task fails to parse the above query when I run the >> > workflow. Is this because metout-config is not creating the Output.cas >> > file? And, what is the best way to specify metadata files for a group of >> > files or for a folder? >> > >> > Thanks, >> > Sheryl >> >> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Chris Mattmann, Ph.D. >> Senior Computer Scientist >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >> Office: 171-266B, Mailstop: 171-246 >> Email: chris.a.mattmann@nasagov >> >> WWW: http://sunset.usc.edu/~mattmann/ >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> Adjunct Assistant Professor, Computer Science Department >> University of Southern California, Los Angeles, CA 90089 USA >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> >> >> >> -- >> -Sheryl > > > > -- > -Sheryl ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
