Thanks Sheryl. OK, that tells me that the met you are expecting to be cataloged isn't.
Can you please post the relevant portions of your product-types.xml, product-type-element-map.xml, and elements.xml to the list? You might want to use pastebin, b/c I don't think the list takes attachments. Cheers, Chris On Sep 15, 2011, at 10:19 PM, Sheryl John wrote: > I tried out for a couple of files, and the metdump alias is not dumping out > any metadata for the respective ProductIds. > > I also tried dumping out metadata for other files for which I've not > specified metout-configs, and they seem to be empty too. > The fmquery works fine and it dumps out all the ProductIds. > > Thanks, > > On Thu, Sep 15, 2011 at 9:08 PM, Mattmann, Chris A (388J) > <[email protected]> wrote: > Hey Sheryl, > > Are you sure that RecordID is being recorded for the particular products that > you are trying to look up? > > One way to find this out: > > 1. get the product IDs back for the products you are interested in. One way > to do this would be > to remove RecordID from your query as you mention below and then add > CAS.ProductId to the > SELECT list, and to change FORMAT='$FileLocation...' to > FORMAT='$CAS.ProductId'. > > 2. Then, for each of the product IDs, check out OODT-306 [1] and use the > metdump alias to dump > out the metadata for each product ID to the current working directory. Then, > inspect that .met file. > Is RecordID recorded? > > Let's start there. > > Cheers, > Chris > > [1] http://issues.apache.org/jira/browse/OODT-306 > > On Sep 15, 2011, at 8:53 PM, Sheryl John wrote: > > > Hi, > > > > I am not clear about the metadata in .met files generated by the > > met-config.xml files. Particularly, where do they fall in the product-type > > and element mappings? > > > > Because in my query, SQL(FORMAT='$FileLocation/$Filename'){ SELECT > > FileLocation,Filename,ISMTable,RecordID FROM ISMRawData WHERE ISMTable = > > 'Chartevents' AND RecordID = '[PID]' , I'm selecting elements that are > > mapped to the product-type, ISMRawData and elements that are mapped to the > > GenericFile product-type. > > > > Except RecordID, which was defined as a key in the metout-config file and > > I'm assuming that the parsing error is because the RecordID is not mapped > > to any product-type. This query works and pulls all the files I need when I > > remove RecordID from the query. Again, since I'm not clear about metadata > > mapping , it's highly likely that I'm wrong about this. > > > > Thanks, > > On Thu, Sep 15, 2011 at 11:17 AM, holenoter <[email protected]> wrote: > > hey Sheryl, > > > > I think there is a misunderstanding on how the met-config.xml files are > > used... the metadata in these files are not added to your workflow context > > metadata... they are only used to create the *.met files for the data file > > for filemgr ingest... about your query I'm not quite clear on how you are > > wanting go use it... it kinda seems like you want to dynamically replace > > out RecordIO... in which case you're gonna need it to look more like > > RecordIO = '[PID]' > > > > -brian > > > > > > On Sep 14, 2011, at 09:18 PM, Sheryl John <[email protected]> wrote: > > > >> Hi Chris, > >> > >> Thanks for pointing that out. I was working on a different query and > >> similar to the one I described. Though I've used only single quotes now, > >> I'm still getting the same parsing error. > >> > >> But, I think what I intended to do with context metadata ( for my > >> workflow) doesn't make sense for a single file product and rather, would > >> apply for a group of files. > >> I guess I have revise my workflow logic and metadata use before getting > >> back to this query. > >> > >> Thanks! > >> > >> > >> > >> On Wed, Sep 14, 2011 at 8:42 PM, Mattmann, Chris A (388J) > >> <[email protected]> wrote: > >> Hi Sheryl, > >> > >> I notice you're using double quotes on the RecordID = "PID", yet you are > >> using single quotes on ISMTable = 'Chartevents'. > >> > >> Try using single quotes on both and see if that fixes it. > >> > >> Thanks! > >> > >> Cheers, > >> Chris > >> > >> > >> On Sep 14, 2011, at 7:03 PM, Sheryl John wrote: > >> > >> > Hi, > >> > > >> > I have defined some key-val pairs for a file( say Output.csv) in a > >> > metout-config.xml for my PGETask Workflow. However, after executing the > >> > workflow, the met-config.xml is not creating a Output.csv.cas file. > >> > > >> > I want to be able to use the above keys/metadata later on in an SQL-like > >> > query from the pgeconfig file. For example, if I've defined 'RecordID' > >> > as a key in the metout-config.xml, I would want to use this metadata in > >> > the following query: > >> > > >> > SQL(FORMAT='$FileLocation/$Filename'){ SELECT > >> > FileLocation,Filename,ISMTable,RecordID FROM ISMRawData WHERE ISMTable = > >> > 'Chartevents' AND RecordID = "PID"} The others keys included in the > >> > query above are elements and product-types that were defined during > >> > ingestion in the File Manager. > >> > > >> > At present, the task fails to parse the above query when I run the > >> > workflow. Is this because metout-config is not creating the Output.cas > >> > file? And, what is the best way to specify metadata files for a group > >> > of files or for a folder? > >> > > >> > Thanks, > >> > Sheryl > >> > >> > >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> Chris Mattmann, Ph.D. > >> Senior Computer Scientist > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > >> Office: 171-266B, Mailstop: 171-246 > >> Email: chris.a.mattmann@nasagov > >> > >> WWW: http://sunset.usc.edu/~mattmann/ > >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> Adjunct Assistant Professor, Computer Science Department > >> University of Southern California, Los Angeles, CA 90089 USA > >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >> > >> > >> > >> > >> -- > >> -Sheryl > > > > > > > > -- > > -Sheryl > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailstop: 171-246 > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Assistant Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > -- > -Sheryl ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
