Thanks to Brian, Chris, Cecilia, Rishi, and Cameron for your feedback on this issue.
Cheers, Mike On 5/23/12 10:13 AM, "Mattmann, Chris A (388J)" <[email protected]> wrote: >Thanks Brian good job tracking this down. > >Thanks to Mike for finding it out -- I will head over to OODT-460 and >think about >the issue there. > >Mike, all: the stuff I showed Mike below about logging.properties and >quoteFields >though is a good illustration of how to see what SQL is being printed by >the FM >catalog and a good way/gotcha FAQ regarding if you have an error related >to numerics and strings if you are combining XMLValidationLayer and >DataSourceCatalog >(for everything but MySQL apparently ;) ). > >Cheers, >Chris > >On May 23, 2012, at 10:10 AM, Brian Foster wrote: > >> hey chris, >> >> this is an SQL compatiblity thing... MySQL doesn't support INTERSECT... >>i created an issue for this fix: (OODT-460) DataSourceCatalog doesn't >>work with MySQL >> >> -brian >> >> On May 22, 2012, at 07:55 PM, "Mattmann, Chris A (388J)" >><[email protected]> wrote: >> >>> Hi Mike, >>> >>> This is where quoteFields comes in. >>> >>> If you are using a combination of XMLValidationLayer with >>>DataSourceCatalog, rather than >>> the element IDs being numeric, they are strings. >>> >>> That's Brian is talking about I think. I you mentioned you were using >>>quoteFields so >>> it should take care of this, but seeing the SQL you're generating can >>>help. >>> >>> Can you edit etc/logging.properties and turn the catalog level WAY up >>>like to ALL? >>> >>> Then restart FM and tell us the queries that are printing from the >>>logs/cas_filemgrX.log.Y. >>> >>> Cheers, >>> Chris >>> >>> On May 22, 2012, at 8:32 AM, Cayanan, Michael D (388J) wrote: >>> >>> > Hey Brian, >>> > >>> > element_id is set to a varchar. I'm using the attached shell script >>>to create the database tables. Since I'm using XMLValidationLayer, does >>>this mean that I should be creating a different set of tables? If so, >>>what would those be? I'm assuming the provided >>>cas-filemgr-schema-mysql.sql is used only if you're using >>>DataSourceValidationLayer... >>> > >>> > -Mike >>> > >>> > From: Brian Foster <[email protected]> >>> > Reply-To: <[email protected]> >>> > Date: Mon, 21 May 2012 22:32:30 +0000 >>> > To: <[email protected]> >>> > Subject: Re: Query Tool Bugs? >>> > >>> > >>> > hey mike, >>> > >>> > i copied in your etc files and policy files into a newly deployed >>>trunk filemgr deployment... i then just commented out the code which >>>talked to sql driver so i could see the SQL which would be sent to the >>>sql driver... basically by doing that i've narrowed it down to >>>something with your SQL connection... did you change element_id to a >>>varchar instead of and int since you are using XMLValidationLayer... >>>the provide cas-filemgr-schema-mysql.sql is setup to work with >>>DataSourceValidationLayer... i think you are getting an error because >>>there are quotes around your element_id and the database is expecting a >>>integer (the quotes are correct in this case though, you DB is just >>>configured wrong) >>> > >>> > -brian >>> > >>> > On May 21, 2012, at 02:42 PM, "Cayanan, Michael D (388J)" >>><[email protected]> wrote: >>> > >>> >> Hey Brian, >>> >> >>> >> Hmmm...interesting. >>> >> >>> >> I believe I attached a filemgr.properties in an earlier e-mail. Did >>>you use this in your configuration as well? >>> >> >>> >> Also, which version of the filemgr are you using on your computer? >>>The trunk? Or the wengine-branch that you pointed me to earlier? >>> >> >>> >> I'm also using a mysql DB with the file manager. I don't think that >>>should matter though in this case, but I could be wrong. I'm assuming >>>you set up your file manager with some other db backend? >>> >> >>> >> Would it be possible to send me a tar package of your file manger >>>deployment that you have on your computer? I want to see if I can just >>>untar it on my computer, start it up and see if that works. >>> >> >>> >> Thanks again for all your help on this, >>> >> Mike >>> >> >>> >> From: Brian Foster <[email protected]> >>> >> Reply-To: <[email protected]> >>> >> Date: Mon, 21 May 2012 19:12:09 +0000 >>> >> To: <[email protected]> >>> >> Subject: Re: Query Tool Bugs? >>> >> >>> >> >>> >> hey mike... i'm kinda stumped... used your configuration on a >>>filemgr on my computer and i get the following output: >>> >> >>> >> $ ./query_tool --url http://localhost:9000 --sql -query "SELECT * >>>FROM L0a_Radar WHERE RangeBeginningDate > '2007-01-01' AND >>>RangeBeginningTime > '12:00:00.000'" >>> >> >>> >> q=AND(RangeBeginningTime:[12:00:00.000 TO null], >>>RangeBeginningDate:[2007-01-01 TO null]) >>> >> May 21, 2012 12:09:58 PM >>>org.apache.oodt.cas.filemgr.catalog.DataSourceCatalog paginateQuery >>> >> INFO: catalog query: executing: (SELECT DISTINCT product_id FROM >>>L0a_Radar_metadata WHERE element_id = >>>'urn:smap:product:RangeBeginningTime' AND metadata_value > >>>'12:00:00.000') INTERSECT (SELECT DISTINCT product_id FROM >>>L0a_Radar_metadata WHERE element_id = >>>'urn:smap:product:RangeBeginningDate' AND metadata_value > >>>'2007-01-01') ORDER BY product_id DESC >>> >> May 21, 2012 12:09:58 PM >>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManager complexQuery >>> >> INFO: Query returned 0 results >>> >> >>> >> There must be something special about your setup... >>> >> >>> >> -brian >>> >> >>> >> On May 21, 2012, at 11:27 AM, Brian Foster <[email protected]> >>>wrote: >>> >> >>> >>> >>> >>> Hey mike... i should have some time today to take a look into this >>>more than just throwing out guesses... i put a print statement in the >>>QueryTool code to see what the SqlParser was doing with the query... >>> >>> >>> >>> i ran the following: >>> >>> ./query_tool --url http://localhost:9000 --sql -query "SELECT * >>>FROM L0a_Radar WHERE RangeBeginningDate > '2007-01-01' AND >>>RangeBeginningTime > '12:00:00.000'" >>> >>> >>> >>> I got the following file manager query: >>> >>> q=AND(RangeBeginningTime:[12:00:00.000 TO null], >>>RangeBeginningDate:[2007-01-01 TO null]) >>> >>> >>> >>> so it appears that it doesn't have anything to do with shell >>>parsing (like i initially thought) or SqlParser... i will take a closer >>>look at your configuration today and let you know what i find. >>> >>> >>> >>> also something that has been know to cause problems in the past is >>>shell aliases... maybe take a look at those if you have any... in the >>>mean time i will try and get to the bottom of this. >>> >>> >>> >>> -brian >>> >>> >>> >>> On May 21, 2012, at 10:02 AM, "Cayanan, Michael D (388J)" >>><[email protected]> wrote: >>> >>> >>> >>>> Hey Brian, >>> >>>> >>> >>>> Yep, I tried the query with spaces before and after > and also >>>surrounded each value with a single quote and it didn't work for me. I >>>also put the query into the shell script as you've suggested and still >>>no luck. >>> >>>> >>> >>>> The exception I'm getting is this: >>> >>>> >>> >>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: >>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed >>>to perform complex query : You have an error in your SQL syntax; check >>>the manual that corresponds to your MySQL server version for the right >>>syntax to use near 'INTERSECT (SELECT DISTINCT product_id FROM >>>L0a_Radar_metadata WHERE element_id =' at line 1 >>> >>>> at >>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcCl >>>ientResponseProcessor.java:104) >>> >>>> at >>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcCli >>>entResponseProcessor.java:71) >>> >>>> at >>>org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) >>> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) >>> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) >>> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) >>> >>>> at >>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery( >>>XmlRpcFileManagerClient.java:945) >>> >>>> at >>>org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.ja >>>va:250) >>> >>>> at >>>org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:241) >>> >>>> Exception in thread "main" >>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: >>>java.lang.Exception: >>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed >>>to perform complex query : You have an error in your SQL syntax; check >>>the manual that corresponds to your MySQL server version for the right >>>syntax to use near 'INTERSECT (SELECT DISTINCT product_id FROM >>>L0a_Radar_metadata WHERE element_id =' at line 1 >>> >>>> at >>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery( >>>XmlRpcFileManagerClient.java:951) >>> >>>> at >>>org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryTool.ja >>>va:250) >>> >>>> at >>>org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:241) >>> >>>> Caused by: org.apache.xmlrpc.XmlRpcException: >>>java.lang.Exception: >>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed >>>to perform complex query : You have an error in your SQL syntax; check >>>the manual that corresponds to your MySQL server version for the right >>>syntax to use near 'INTERSECT (SELECT DISTINCT product_id FROM >>>L0a_Radar_metadata WHERE element_id =' at line 1 >>> >>>> at >>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcCl >>>ientResponseProcessor.java:104) >>> >>>> at >>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcCli >>>entResponseProcessor.java:71) >>> >>>> at >>>org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) >>> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) >>> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) >>> >>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) >>> >>>> at >>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery( >>>XmlRpcFileManagerClient.java:945) >>> >>>> ... 2 more >>> >>>> >>> >>>> Attached is the query tool script, which contains the query I'm >>>trying. It appears that this is a bug. Since query tool has been >>>deprecated in the trunk in favor of the super new file manager client >>>tool, then I'll go ahead and file a JIRA issue for this against the >>>file manager. >>> >>>> >>> >>>> In the meantime, I'll definitely look more into this and see if I >>>can find out where the bug is at. >>> >>>> >>> >>>> Thanks for your help! >>> >>>> >>> >>>> -Mike >>> >>>> >>> >>>> From: Brian Foster <[email protected]> >>> >>>> Reply-To: <[email protected]> >>> >>>> Date: Fri, 18 May 2012 00:51:07 +0000 >>> >>>> To: <[email protected]> >>> >>>> Subject: Re: Query Tool Bugs? >>> >>>> >>> >>>> >>> >>>> hey mike, >>> >>>> >>> >>>> You putting a space between the operators? its hard to tell in >>>emails, stuff gets normalized sometimes. >>> >>>> >>> >>>> RangeBeginningDate > '2007-01-01' >>> >>>> >>> >>>> Should be a space before and after > >>> >>>> >>> >>>> each value should start and end with single quote. >>> >>>> >>> >>>> -brian >>> >>>> >>> >>>> On May 17, 2012, at 04:18 PM, "Cayanan, Michael D (388J)" >>><[email protected]> wrote: >>> >>>> >>> >>>>> Hey Brian, >>> >>>>> >>> >>>>> I checked out the wengine-branch filemgr and unfortunately, it >>>still didn't work. I tried a query using the wengine branch's >>>query_tool since this version of the filemgr-client didn't have the >>>ability to query. >>> >>>>> >>> >>>>> In any case, I took the liberty of diving into the Query Tool >>>code to see if I can find out more as to what is going on. So this is >>>what I found: >>> >>>>> >>> >>>>> Surrounding single quotes around each of the conditions got me >>>further into the code. So, when I entered the query as follows, >>> >>>>> >>> >>>>> SELECT * FROM L0a_Radar WHERE 'RangeBeginningDate>2007-01-01' >>>AND 'RangeBeginningTime>12:00:00Z' >>> >>>>> >>> >>>>> The SQLParser class liked the syntax and it produced the >>>following post fix statement: >>> >>>>> >>> >>>>> Post Fix Statement: ['RangeBeginningDate>2007-01-01', >>>'RangeBeginningTime>12:00:00Z', AND] >>> >>>>> >>> >>>>> I think from here, it creates some Boolean Query Criteria with >>>the following content: >>> >>>>> >>> >>>>> Boolean Query Criteria: AND('RangeBeginningTime:[12:00:00Z' TO >>>null], 'RangeBeginningDate:[2007-01-01' TO null]) >>> >>>>> >>> >>>>> I noticed that there are unmatched single quotes around the date >>>and time values. Not sure how that came about. The tool still threw an >>>exception. I'm guessing it's because of the unmatched single quotes. >>> >>>>> >>> >>>>> I tried surrounding the date and time values with single quotes >>>as well, but the SQLParser class didn't like that either: >>> >>>>> >>> >>>>> SELECT * FROM L0a_Radar WHERE 'RangeBeginningDate>'2007-01-01'' >>>AND 'RangeBeginningTime>'12:00:00Z'' >>> >>>>> >>> >>>>> >>> >>>>> -Mike >>> >>>>> >>> >>>>> From: Brian Foster <[email protected]> >>> >>>>> Reply-To: <[email protected]> >>> >>>>> Date: Thu, 17 May 2012 18:40:20 +0000 >>> >>>>> To: <[email protected]> >>> >>>>> Subject: Re: Query Tool Bugs? >>> >>>>> >>> >>>>> >>> >>>>> hey mike, >>> >>>>> >>> >>>>> would you mind trying to run this query using the wengine-branch >>>filemgr >>>(https://svn.apache.org/repos/asf/oodt/branches/wengine-branch/filemgr/) >>>... there are a lot of bug fixes floating around in both Lucene and >>>DataSource Catalogs in that branch... if it works with that then we >>>know it is a bug with trunk. >>> >>>>> >>> >>>>> -brian >>> >>>>> >>> >>>>> On May 16, 2012, at 07:26 AM, "Cayanan, Michael D (388J)" >>><[email protected]> wrote: >>> >>>>> >>> >>>>>> Hey Brian, >>> >>>>>> >>> >>>>>> Ya, I tried the query with and without the double quotes >>>surrounding the where clause and they both didn't work for me. >>> >>>>>> >>> >>>>>> In any case, I've built the latest file manager in the trunk >>>and it fixed that StringIndexOutofBoundsException issue I was seeing. >>>Thanks! >>> >>>>>> >>> >>>>>> Out of curiosity, I tried using the latest File Manager Client >>>tool with the query with multiple conditions to see if I could get a >>>result, but it threw an exception as well: >>> >>>>>> >>> >>>>>> ./filemgr-client -u http://localhost:9000 -q "SELECT * FROM >>>L0a_Radar WHERE RangeBeginningDate>'2007-01-01' AND >>>RangeBeginningTime>'12:00:00Z'" -op -sqlQuery >>> >>>>>> May 16, 2012 7:20:53 AM >>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient <init> >>> >>>>>> INFO: Loading File Manager Configuration Properties from: >>>[../etc/filemgr.properties] >>> >>>>>> org.apache.xmlrpc.XmlRpcException: java.lang.Exception: >>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed >>>to perform complex query : You have an error in your SQL syntax; check >>>the manual that corresponds to your MySQL server version for the right >>>syntax to use near 'INTERSECT (SELECT DISTINCT product_id FROM >>>L0a_Radar_metadata WHERE element_id =' at line 1 >>> >>>>>> at >>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeException(XmlRpcCl >>>ientResponseProcessor.java:104) >>> >>>>>> at >>>org.apache.xmlrpc.XmlRpcClientResponseProcessor.decodeResponse(XmlRpcCli >>>entResponseProcessor.java:71) >>> >>>>>> at >>>org.apache.xmlrpc.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:73) >>> >>>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:194) >>> >>>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:185) >>> >>>>>> at org.apache.xmlrpc.XmlRpcClient.execute(XmlRpcClient.java:178) >>> >>>>>> at >>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQuery( >>>XmlRpcFileManagerClient.java:990) >>> >>>>>> at >>>org.apache.oodt.cas.filemgr.cli.action.AbstractQueryCliAction.execute(Ab >>>stractQueryCliAction.java:75) >>> >>>>>> at >>>org.apache.oodt.cas.cli.CmdLineUtility.execute(CmdLineUtility.java:331) >>> >>>>>> at >>>org.apache.oodt.cas.cli.CmdLineUtility.run(CmdLineUtility.java:187) >>> >>>>>> at >>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.main(XmlRpcFi >>>leManagerClient.java:1323) >>> >>>>>> ERROR: Failed to perform sql query : sortBy 'null', >>>outputFormat 'null', and delimiter ' >>> >>>>>> ', filterAlgor 'null', startDateTimeMetKey 'null', >>>endDateTimeMetKey 'null', priorityMetKey 'null', null' : >>>java.lang.Exception: >>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: Failed >>>to perform complex query : You have an error in your SQL syntax; check >>>the manual that corresponds to your MySQL server version for the right >>>syntax to use near 'INTERSECT (SELECT DISTINCT product_id FROM >>>L0a_Radar_metadata WHERE element_id =' at line 1 >>> >>>>>> >>> >>>>>> Let me know if you'd like for me to file a bug for this. >>> >>>>>> >>> >>>>>> Thanks, >>> >>>>>> Mike >>> >>>>>> >>> >>>>>> From: Brian Foster <[email protected]> >>> >>>>>> Reply-To: <[email protected]> >>> >>>>>> Date: Mon, 14 May 2012 22:28:58 +0000 >>> >>>>>> To: <[email protected]> >>> >>>>>> Subject: Re: Query Tool Bugs? >>> >>>>>> >>> >>>>>> hey mike, >>> >>>>>> >>> >>>>>> first pass over the policy files look good... however, is the >>>way you posted the query the way you are running it? >>> >>>>>> >>> >>>>>>> ./query_tool --url http://localhost:9000 --sql -query "SELECT >>>* FROM L0a_Radar WHERE "RangeBeginningDate>'2007-01-01' AND >>>RangeBeginningTime>'12:00:00.000Z'"" >>> >>>>>> >>> >>>>>> you have an extra set of double quotes around the where >>>clause... try removing those >>> >>>>>> >>> >>>>>> -brian >>> >>>>>> >>> >>>>>> On May 14, 2012, at 09:32 AM, "Cayanan, Michael D (388J)" >>><[email protected]> wrote: >>> >>>>>> >>> >>>>>>> Hey Chris, >>> >>>>>>> >>> >>>>>>> On 5/12/12 11:25 AM, "Mattmann, Chris A (388J)" >>> >>>>>>> <[email protected]> wrote: >>> >>>>>>> >>> >>>>>>> >Hey Mike, >>> >>>>>>> > >>> >>>>>>> >On May 11, 2012, at 6:07 AM, Cayanan, Michael D (388J) wrote: >>> >>>>>>> > >>> >>>>>>> >>>> >>> >>>>>>> >>>> >>>org.apache.oodt.cas.filemgr.structs.exceptions.CatalogException: >>> >>>>>>> >>>>Failed >>> >>>>>>> >>>> to perform complex query : You have an error in your SQL >>>syntax; check >>> >>>>>>> >>>> the manual that corresponds to your MySQL server version >>>for the right >>> >>>>>>> >>>> syntax to use near 'INTERSECT (SELECT DISTINCT product_id >>>FROM >>> >>>>>>> >>>> L0a_Radar_metadata WHERE element_id =' at line 1 >>> >>>>>>> >>>> at >>> >>>>>>> >>>> >>> >>>>>>> >>>>>>>org.apache.oodt.cas.filemgr.system.XmlRpcFileManagerClient.complexQu >>>>>>>ery >>> >>>>>>> >>>>(X >>> >>>>>>> >>>> mlRpcFileManagerClient.java:958) >>> >>>>>>> >>>> at >>> >>>>>>> >>>> >>> >>>>>>> >>>>>>>org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryToo >>>>>>>l.j >>> >>>>>>> >>>>av >>> >>>>>>> >>>> a:251) >>> >>>>>>> >>>> at >>> >>>>>>> >>>>>>>org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:241) >>> >>>>>>> >>> >>> >>>>>>> >>> Just out of curiosity, is that correct ISO 8601 date/time >>>format? Looks >>> >>>>>>> >>> like a partial one, missing the timezone do you think that >>>might >>> >>>>>>> >>> affect ir? >>> >>>>>>> >> >>> >>>>>>> >> I talked with Rishi regarding this and he recommended that >>>the date and >>> >>>>>>> >> time be split when performing a query. Reason being is that >>>the query >>> >>>>>>> >>tool >>> >>>>>>> >> blows up when trying to compare datetime values. He >>>mentioned that he >>> >>>>>>> >> tried querying against ISO 8601 date/time values before and >>>it didn't >>> >>>>>>> >>work >>> >>>>>>> >> for him and the way around it was to split it up. I think >>>behind the >>> >>>>>>> >> scenes, the query tool is actually doing an ascii >>>comparison, which >>> >>>>>>> >>might >>> >>>>>>> >> be why the tool might be having performance issues? >>> >>>>>>> > >>> >>>>>>> >Gotcha, that might help, yes. I was thinking: what is your >>>repository >>> >>>>>>> >manager, >>> >>>>>>> >and catalog combination? If you are using e.g., a >>>DataSourceCatalog, >>> >>>>>>> >with the XMLRepositoryManager, you'll need to turn on the >>>quoteFields >>> >>>>>>> >option in the filemgr.properties for the DataSourceCatalog. >>>This is >>> >>>>>>> >because, >>> >>>>>>> >in these scenarios, the identifier for elementIds is a >>>string, compared >>> >>>>>>> >to a >>> >>>>>>> >number (which would be the case if you used the >>> >>>>>>> >DataSourceRepositoryManager -- >>> >>>>>>> >the short answer there is don't, it's not as well maintained >>>as the XML >>> >>>>>>> >one). >>> >>>>>>> >>> >>>>>>> My repository manager and catalog combination is defined as >>>follows in the >>> >>>>>>> filemgr.properties: >>> >>>>>>> >>> >>>>>>> >>>filemgr.repository.factory=org.apache.oodt.cas.filemgr.repository.XMLRep >>>osi >>> >>>>>>> toryManagerFactory >>> >>>>>>> >>> >>>>>>> >>>filemgr.catalog.factory=org.apache.oodt.cas.filemgr.catalog.DataSourceCa >>>tal >>> >>>>>>> ogFactory >>> >>>>>>> >>> >>>>>>> org.apache.oodt.cas.filemgr.catalog.datasource.quoteFields=true >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> > >>> >>>>>>> >> >>> >>>>>>> >>> >>> >>>>>>> >>>> >>> >>>>>>> >>>> I tried surrounding the entire condition with quotes, but >>>still no >>> >>>>>>> >>>>luck: >>> >>>>>>> >>>> >>> >>>>>>> >>>> ./query_tool --url http://localhost:9000 --sql -query >>>"SELECT * FROM >>> >>>>>>> >>>> L0a_Radar WHERE "RangeBeginningDate>'2007-01-01' AND >>> >>>>>>> >>>> RangeBeginningTime>'12:00:00.000Z'"" >>> >>>>>>> >>>> Ambiguous output redirect. >>> >>>>>>> >>>> >>> >>>>>>> >>>> I'm assuming this is a syntax thing, although I don't >>>know what the >>> >>>>>>> >>>> tool is expecting. >>> >>>>>>> >>> >>> >>>>>>> >>> Did you check the code in SVN? >>> >>>>>>> >> >>> >>>>>>> >> I'm running 0.3 of the code. Does the trunk fix this? I >>>have the code >>> >>>>>>> >> checked out onto my local machine. I can certainly build >>>the trunk and >>> >>>>>>> >>see >>> >>>>>>> >> if I get the same results. >>> >>>>>>> > >>> >>>>>>> >I think there is a fix for something similar to this in the >>>trunk (as >>> >>>>>>> >bfoster mentioned), >>> >>>>>>> >but thinking about this more, I bet you're having the >>>quoteFields >>> >>>>>>> >problem, per >>> >>>>>>> >above. Scope it out and let me know. >>> >>>>>>> >>> >>>>>>> Based on the properties specified above, the quoteFields >>>option is turned >>> >>>>>>> on. So I think the properties are set correctly, no? >>> >>>>>>> It'll be interesting to see if Brian was able to find an error >>>in the >>> >>>>>>> policy files that I sent in the previous e-mail. >>> >>>>>>> >>> >>>>>>> > >>> >>>>>>> >> >>> >>>>>>> >>> >>> >>>>>>> >>>> >>> >>>>>>> >>>> My second issue that I'm running into is in regards to >>>querying of >>> >>>>>>> >>>> dates. I tried the following query below and got the >>>following output: >>> >>>>>>> >>>> >>> >>>>>>> >>>> ./query_tool --url http://localhost:9000 --sql -query >>>"SELECT * FROM >>> >>>>>>> >>>> L0a_Radar WHERE RangeBeginningDate>'2007-03-02'" >>> >>>>>>> >>>> log4j:WARN No appenders could be found for logger >>> >>>>>>> >>>> (org.apache.commons.httpclient.HttpClient). >>> >>>>>>> >>>> log4j:WARN Please initialize the log4j system properly. >>> >>>>>>> >>>> log4j:WARN See >>>http://logging.apache.org/log4j/1.2/faq.html#noconfig >>> >>>>>>> >>>> for more info. >>> >>>>>>> >>>> Exception in thread "main" >>>java.lang.StringIndexOutOfBoundsException: >>> >>>>>>> >>>> String index out of range: -1 >>> >>>>>>> >>>> at >>> >>>>>>> >>>> >>> >>>>>>> >>>>>>>java.lang.AbstractStringBuilder.substring(AbstractStringBuilder.java >>>>>>>:88 >>> >>>>>>> >>>>1) >>> >>>>>>> >>>> at java.lang.StringBuffer.substring(StringBuffer.java:416) >>> >>>>>>> >>>> at >>> >>>>>>> >>>> >>> >>>>>>> >>>>>>>org.apache.oodt.cas.filemgr.tools.QueryTool.performSqlQuery(QueryToo >>>>>>>l.j >>> >>>>>>> >>>>av >>> >>>>>>> >>>> a:255) >>> >>>>>>> >>>> at >>> >>>>>>> >>>>>>>org.apache.oodt.cas.filemgr.tools.QueryTool.main(QueryTool.java:241) >>> >>>>>>> >>>> >>> >>>>>>> >>>> For this particular product, I have 1 product in my >>>catalog where the >>> >>>>>>> >>>> RangeBeginningDate is equal to '2007-03-01'. Not sure if >>>that factors >>> >>>>>>> >>>> into why an exception is being thrown here. When I use an >>>earlier date >>> >>>>>>> >>>> on my query, the tool returns a result as expected: >>> >>>>>>> >>>> >>> >>>>>>> >>>> ./query_tool --url http://localhost:9000 --sql -query >>>"SELECT * FROM >>> >>>>>>> >>>> L0a_Radar WHERE RangeBeginningDate>'2007-01-01'" >>> >>>>>>> >>>> log4j:WARN No appenders could be found for logger >>> >>>>>>> >>>> (org.apache.commons.httpclient.HttpClient). >>> >>>>>>> >>>> log4j:WARN Please initialize the log4j system properly. >>> >>>>>>> >>>> log4j:WARN See >>>http://logging.apache.org/log4j/1.2/faq.html#noconfig >>> >>>>>>> >>>> for more info. >>> >>>>>>> >>>> >>> >>>>>>> >>>> >>> >>>>>>> >>>>>>>/Users/mcayanan/smap/staging,2007-03-01,23:30:25.000Z,314,L0a_Radar, >>>>>>>V20 >>> >>>>>>> >>>>51 >>> >>>>>>> >>>> >>> >>>>>>> >>>>>>>7SGS0706023302501.VCD,V20517SGS0706023302501.VCD,2012-05-08T14:27:59 >>>>>>>.38 >>> >>>>>>> >>>>5- >>> >>>>>>> >>>> 07:00,L0a_Radar,23:30:25.000Z,2007-03-01 >>> >>>>>>> >>> >>> >>>>>>> >>> Interesting! Did you scope the code to see if there's a >>>RangeQuery >>> >>>>>>> >>>issue? >>> >>>>>>> >>> >>> >>>>>>> >>> Feel free to file a bug and would love you to investigate! >>> >>>>>>> >> >>> >>>>>>> >> I haven't dived into the code, but will certainly do this >>>as SMAP will >>> >>>>>>> >> need these capabilities. I will file a bug if it turns out >>>that this is >>> >>>>>>> >> indeed a bug. >>> >>>>>>> > >>> >>>>>>> >Great Mike, thanks. >>> >>>>>>> > >>> >>>>>>> >Cheers, >>> >>>>>>> >Chris >>> >>>>>>> >>> >>>>>>> Thanks, >>> >>>>>>> Mike >>> >>>>>>> >>> >>>>>>> > >>> >>>>>>> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>>>>>> >Chris Mattmann, Ph.D. >>> >>>>>>> >Senior Computer Scientist >>> >>>>>>> >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>> >>>>>>> >Office: 171-266B, Mailstop: 171-246 >>> >>>>>>> >Email: [email protected] >>> >>>>>>> >WWW: http://sunset.usc.edu/~mattmann/ >>> >>>>>>> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>>>>>> >Adjunct Assistant Professor, Computer Science Department >>> >>>>>>> >University of Southern California, Los Angeles, CA 90089 USA >>> >>>>>>> >>>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>>>>>> > >>> >>>>>>> >>> >>>> #!/bin/sh >>> >>>> >>> >>>> $JAVA_HOME/bin/java -Djava.ext.dirs=../lib >>>org.apache.oodt.cas.filemgr.tools.QueryTool --url http://localhost:9000 >>>--sql -query "SELECT * FROM L0a_Radar WHERE RangeBeginningDate > >>>'2007-01-01' AND RangeBeginningTime > '12:00:00.000'" >>> > <createProductTypeSQL.sh> >>> >>> >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Chris Mattmann, Ph.D. >>> Senior Computer Scientist >>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >>> Office: 171-266B, Mailstop: 171-246 >>> Email: [email protected] >>> WWW: http://sunset.usc.edu/~mattmann/ >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> Adjunct Assistant Professor, Computer Science Department >>> University of Southern California, Los Angeles, CA 90089 USA >>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> > > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Senior Computer Scientist >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 171-266B, Mailstop: 171-246 >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Adjunct Assistant Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >
