Re: Derby Key Words
On 11.12.2012 23:16, david myers wrote: > need to worry about this ? is simply listing them in my < > getDerbyKeyWordsTest_app.properties > going to be Just for your FYI, the *_app.properties stuff is part of our old testing harness, nowadays we build tests in JUnit only. The test directories contain a mix of old test harness tests (we're always trying to convert the rest of those) and JUnit tests. Almost all test files named Test.java (note the capital "T") are examples of JUnit based tests. Thanks, Dag
Re: Derby Key Words
Rik, Sounds fair to me. David. On 12/12/12 14:30, Rick Hillegas wrote: Hi David, Thanks for continuing to think about this issue. Dag's solution looks good enough to me. I don't see any compelling reason to also expose this functionality via a second api, namely, a Derby-specific extension to the JDBC metadata. I'm glad that you want to contribute to Derby. At this point I think that a better introductory task would be to pick one of the Derby JIRA issues which is marked "Newcomer". Thanks, -Rick On 12/11/12 2:16 PM, david myers wrote: On 04/12/12 22:33, Dag Wanvik wrote: On 04.12.2012 17:57, Rick Hillegas wrote: Hi David, I have added a comment to DERBY-3256. I think that Dag's solution looks complete and reasonable. I recommend that it be committed and documented. Does his solution not solve your original problem? Thanks for looking at the patch, Rick. I think David wants to extend this solution so one could get these keywords via DataBaseMetaData? Thanks, Dag Hi Rick and Dag, Dag's solution is exceedingly complete, but like Dag mentions I am putting the finishing touches on a DataBaseMetaData method to kick out the info (in the form of a table as per your original suggestion). Currently it seems to 'work', but I have a few 'niggles' that I want to solve, mainly revolving around the test(s) I'm writing. Currently my metadata method is in its own class that inherits from the 'proper' one, so no danger of me breaking something else on the way (eventually if its good we can simply copy / paste it back into the parent). The same is true for my test class. Also I'm not sure how to test for the presence of the JJ file and Dag's patch on which I obviously rely, I understand (sort of) how Dag uses the build paths to find the file in his patch, but I'm not sure how to code these into a test, i'm sure they are in a test somewhere, just for the moment I haven't 'seen' them, or do I even need to worry about this ? is simply listing them in my < getDerbyKeyWordsTest_app.properties > going to be sufficient ? again how to I give a location based on the derbyHome or similar in this file? If you can point me to a good example test class that does this sort of thing, I'll work my way through it and use the same model for my test class. Personally I just want to give a good test with sensible responses when they fail if the files I rely on strangely go missing or move to another location. I'll try to get back with the exact problems I seem to be facing, which for the moment are rather varied, and probably linked to my setup of Eclipse... i'm getting there slowly though. I just need to find a few extra hours somewhere... especially as the wife has decided it is time to redecorate the bedroom ;). Speak again soon David Thanks, -Rick On 11/26/12 2:16 PM, david myers wrote: Hello All, As always first off I should appologise for being quiet, i've been reading through the code and I think I start to understand things that were not very clear to me previously. My current situation is now the following I've got myself well sorted out. Eclipse now has a 'handle' on the svn repo (copied locally of course), and I've been able to grab Dag's patch and it to this, all from inside eclipse (which I'm pleased worked out). Dag's code is interesting, very cool. I feel my next steps are as follows... I need to write a new class (or extend an existing one). I guess something in DB MetaData. But what and where? Then I need to test it. Where should I put my test class (or again should I add the test to a class that already exists. I feel that I am now ready to implement this. I just need some pointers of how to run my individual test case (I make the assumption that as I'm not going near anything else in the first instance I shouldn't break it (that is also why I propose a new subclass of one of the existing dbMetaData classes). I hope to become a little more active before christmass David. On 15/10/12 20:48, david myers wrote: Dag, thanks for the info. I'll have a try with that later on. One thing though, should I not be in my < derby trunk sandbox of checked out from svn[1] > before doing wget (surely I'll end up with it in the wrong place)... I'm probably showing my ignorance about wget and svn as I've not done this sort of stuff before ;) Not that it matters, I'll check it out later on David On 15/10/12 13:06, Dag Wanvik wrote: Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build o
Re: Derby Key Words
Hi David, Thanks for continuing to think about this issue. Dag's solution looks good enough to me. I don't see any compelling reason to also expose this functionality via a second api, namely, a Derby-specific extension to the JDBC metadata. I'm glad that you want to contribute to Derby. At this point I think that a better introductory task would be to pick one of the Derby JIRA issues which is marked "Newcomer". Thanks, -Rick On 12/11/12 2:16 PM, david myers wrote: On 04/12/12 22:33, Dag Wanvik wrote: On 04.12.2012 17:57, Rick Hillegas wrote: Hi David, I have added a comment to DERBY-3256. I think that Dag's solution looks complete and reasonable. I recommend that it be committed and documented. Does his solution not solve your original problem? Thanks for looking at the patch, Rick. I think David wants to extend this solution so one could get these keywords via DataBaseMetaData? Thanks, Dag Hi Rick and Dag, Dag's solution is exceedingly complete, but like Dag mentions I am putting the finishing touches on a DataBaseMetaData method to kick out the info (in the form of a table as per your original suggestion). Currently it seems to 'work', but I have a few 'niggles' that I want to solve, mainly revolving around the test(s) I'm writing. Currently my metadata method is in its own class that inherits from the 'proper' one, so no danger of me breaking something else on the way (eventually if its good we can simply copy / paste it back into the parent). The same is true for my test class. Also I'm not sure how to test for the presence of the JJ file and Dag's patch on which I obviously rely, I understand (sort of) how Dag uses the build paths to find the file in his patch, but I'm not sure how to code these into a test, i'm sure they are in a test somewhere, just for the moment I haven't 'seen' them, or do I even need to worry about this ? is simply listing them in my < getDerbyKeyWordsTest_app.properties > going to be sufficient ? again how to I give a location based on the derbyHome or similar in this file? If you can point me to a good example test class that does this sort of thing, I'll work my way through it and use the same model for my test class. Personally I just want to give a good test with sensible responses when they fail if the files I rely on strangely go missing or move to another location. I'll try to get back with the exact problems I seem to be facing, which for the moment are rather varied, and probably linked to my setup of Eclipse... i'm getting there slowly though. I just need to find a few extra hours somewhere... especially as the wife has decided it is time to redecorate the bedroom ;). Speak again soon David Thanks, -Rick On 11/26/12 2:16 PM, david myers wrote: Hello All, As always first off I should appologise for being quiet, i've been reading through the code and I think I start to understand things that were not very clear to me previously. My current situation is now the following I've got myself well sorted out. Eclipse now has a 'handle' on the svn repo (copied locally of course), and I've been able to grab Dag's patch and it to this, all from inside eclipse (which I'm pleased worked out). Dag's code is interesting, very cool. I feel my next steps are as follows... I need to write a new class (or extend an existing one). I guess something in DB MetaData. But what and where? Then I need to test it. Where should I put my test class (or again should I add the test to a class that already exists. I feel that I am now ready to implement this. I just need some pointers of how to run my individual test case (I make the assumption that as I'm not going near anything else in the first instance I shouldn't break it (that is also why I propose a new subclass of one of the existing dbMetaData classes). I hope to become a little more active before christmass David. On 15/10/12 20:48, david myers wrote: Dag, thanks for the info. I'll have a try with that later on. One thing though, should I not be in my < derby trunk sandbox of checked out from svn[1] > before doing wget (surely I'll end up with it in the wrong place)... I'm probably showing my ignorance about wget and svn as I've not done this sort of stuff before ;) Not that it matters, I'll check it out later on David On 15/10/12 13:06, Dag Wanvik wrote: Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build on it. Good luck! Thanks, Dag On 14.10.2012 20:28, david myers wrote: Hi all
Re: Derby Key Words
On 04/12/12 22:33, Dag Wanvik wrote: On 04.12.2012 17:57, Rick Hillegas wrote: Hi David, I have added a comment to DERBY-3256. I think that Dag's solution looks complete and reasonable. I recommend that it be committed and documented. Does his solution not solve your original problem? Thanks for looking at the patch, Rick. I think David wants to extend this solution so one could get these keywords via DataBaseMetaData? Thanks, Dag Hi Rick and Dag, Dag's solution is exceedingly complete, but like Dag mentions I am putting the finishing touches on a DataBaseMetaData method to kick out the info (in the form of a table as per your original suggestion). Currently it seems to 'work', but I have a few 'niggles' that I want to solve, mainly revolving around the test(s) I'm writing. Currently my metadata method is in its own class that inherits from the 'proper' one, so no danger of me breaking something else on the way (eventually if its good we can simply copy / paste it back into the parent). The same is true for my test class. Also I'm not sure how to test for the presence of the JJ file and Dag's patch on which I obviously rely, I understand (sort of) how Dag uses the build paths to find the file in his patch, but I'm not sure how to code these into a test, i'm sure they are in a test somewhere, just for the moment I haven't 'seen' them, or do I even need to worry about this ? is simply listing them in my < getDerbyKeyWordsTest_app.properties > going to be sufficient ? again how to I give a location based on the derbyHome or similar in this file? If you can point me to a good example test class that does this sort of thing, I'll work my way through it and use the same model for my test class. Personally I just want to give a good test with sensible responses when they fail if the files I rely on strangely go missing or move to another location. I'll try to get back with the exact problems I seem to be facing, which for the moment are rather varied, and probably linked to my setup of Eclipse... i'm getting there slowly though. I just need to find a few extra hours somewhere... especially as the wife has decided it is time to redecorate the bedroom ;). Speak again soon David Thanks, -Rick On 11/26/12 2:16 PM, david myers wrote: Hello All, As always first off I should appologise for being quiet, i've been reading through the code and I think I start to understand things that were not very clear to me previously. My current situation is now the following I've got myself well sorted out. Eclipse now has a 'handle' on the svn repo (copied locally of course), and I've been able to grab Dag's patch and it to this, all from inside eclipse (which I'm pleased worked out). Dag's code is interesting, very cool. I feel my next steps are as follows... I need to write a new class (or extend an existing one). I guess something in DB MetaData. But what and where? Then I need to test it. Where should I put my test class (or again should I add the test to a class that already exists. I feel that I am now ready to implement this. I just need some pointers of how to run my individual test case (I make the assumption that as I'm not going near anything else in the first instance I shouldn't break it (that is also why I propose a new subclass of one of the existing dbMetaData classes). I hope to become a little more active before christmass David. On 15/10/12 20:48, david myers wrote: Dag, thanks for the info. I'll have a try with that later on. One thing though, should I not be in my < derby trunk sandbox of checked out from svn[1] > before doing wget (surely I'll end up with it in the wrong place)... I'm probably showing my ignorance about wget and svn as I've not done this sort of stuff before ;) Not that it matters, I'll check it out later on David On 15/10/12 13:06, Dag Wanvik wrote: Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build on it. Good luck! Thanks, Dag On 14.10.2012 20:28, david myers wrote: Hi all, must apologise for being so quiet,busy with my 'real' job etc. However this doesn't mean I've been ignoring what I want to do. I've been reading around the code to determine the files that I am going to require to initiate this little modification. I think I am begining to understand how things work a little better. Firstly however I want to ensure I have well understood. The file sqlgrammar.jj is read by javacc and creates a bunch of other .java clases. SQLParserConstants.java
Re: Derby Key Words
On 04.12.2012 17:57, Rick Hillegas wrote: > Hi David, > > I have added a comment to DERBY-3256. I think that Dag's solution > looks complete and reasonable. I recommend that it be committed and > documented. Does his solution not solve your original problem? > Thanks for looking at the patch, Rick. I think David wants to extend this solution so one could get these keywords via DataBaseMetaData? Thanks, Dag > Thanks, > -Rick > > On 11/26/12 2:16 PM, david myers wrote: >> Hello All, >> >> As always first off I should appologise for being quiet, i've been >> reading through the code and I think I start to understand things >> that were not very clear to me previously. >> >> My current situation is now the following >> >> I've got myself well sorted out. >> >> Eclipse now has a 'handle' on the svn repo (copied locally of >> course), and I've been able to grab Dag's patch and it to this, all >> from inside eclipse (which I'm pleased worked out). >> >> Dag's code is interesting, very cool. >> >> I feel my next steps are as follows... >> >> I need to write a new class (or extend an existing one). I guess >> something in DB MetaData. But what and where? >> Then I need to test it. Where should I put my test class (or again >> should I add the test to a class that already exists. >> >> I feel that I am now ready to implement this. I just need some >> pointers of how to run my individual test case (I make the assumption >> that as I'm not going near anything else in the first instance I >> shouldn't break it (that is also why I propose a new subclass of one >> of the existing dbMetaData classes). >> >> I hope to become a little more active before christmass >> >> David. >> >> >> On 15/10/12 20:48, david myers wrote: >>> Dag, >>> >>> thanks for the info. I'll have a try with that later on. >>> One thing though, should I not be in my < derby trunk sandbox of >>> checked out from svn[1] > before doing wget (surely I'll end up with >>> it in the wrong place)... I'm probably showing my ignorance about >>> wget and svn as I've not done this sort of stuff before ;) >>> >>> Not that it matters, I'll check it out later on >>> >>> David >>> >>> On 15/10/12 13:06, Dag Wanvik wrote: Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build on it. Good luck! Thanks, Dag On 14.10.2012 20:28, david myers wrote: > Hi all, > > must apologise for being so quiet,busy with my 'real' job etc. > However > this doesn't mean I've been ignoring what I want to do. > > I've been reading around the code to determine the files that I am > going to require to initiate this little modification. I think I am > begining to understand how things work a little better. Firstly > however I want to ensure I have well understood. > > The file sqlgrammar.jj is read by javacc and creates a bunch of other > .java clases. > > SQLParserConstants.java is one of these files, it is an interface > that > is implemented by the SQLParser class (also created from > sqlgrammar.jj). > > So if I can modify sqlgrammar.jj I could 'inject' into the SQLParser > class a method that return the list of static final members > (stored in > the tokenImage string array member, so in fact I could just return > this member). > > My problems currently now revolve around the following. > > DAG has created a method that reads the sqlgrammar.jj file directly > and creates the Keywords.java file. I can't seem to find this in my > source tree, is this related to needing the specific DERBY-3256 > checkout. I've tried using the SVN plugin for eclipse to find this > checkout, but so far to no success. Any chance someone could walk me > through doing this from the CLI (ubuntu 12.04). > > Should I mention at this point I'm rather new to SVN (actually > version > control in general!), did I say previously this is my first multi > person project? > sorry I digress > > I've located the files that I think I'm going to need. > EmbededDataBaseMetaData[40].java and TestDBMetaData.java where I will > put my < getDerbyKeyWords() > method and test. > > Rik thanks for waving your hands about the compile time tool. I think > this may be what I need to implement my < getTokenImage() > so any > pointers would be great. >
Re: Derby Key Words
Hi David, I have added a comment to DERBY-3256. I think that Dag's solution looks complete and reasonable. I recommend that it be committed and documented. Does his solution not solve your original problem? Thanks, -Rick On 11/26/12 2:16 PM, david myers wrote: Hello All, As always first off I should appologise for being quiet, i've been reading through the code and I think I start to understand things that were not very clear to me previously. My current situation is now the following I've got myself well sorted out. Eclipse now has a 'handle' on the svn repo (copied locally of course), and I've been able to grab Dag's patch and it to this, all from inside eclipse (which I'm pleased worked out). Dag's code is interesting, very cool. I feel my next steps are as follows... I need to write a new class (or extend an existing one). I guess something in DB MetaData. But what and where? Then I need to test it. Where should I put my test class (or again should I add the test to a class that already exists. I feel that I am now ready to implement this. I just need some pointers of how to run my individual test case (I make the assumption that as I'm not going near anything else in the first instance I shouldn't break it (that is also why I propose a new subclass of one of the existing dbMetaData classes). I hope to become a little more active before christmass David. On 15/10/12 20:48, david myers wrote: Dag, thanks for the info. I'll have a try with that later on. One thing though, should I not be in my < derby trunk sandbox of checked out from svn[1] > before doing wget (surely I'll end up with it in the wrong place)... I'm probably showing my ignorance about wget and svn as I've not done this sort of stuff before ;) Not that it matters, I'll check it out later on David On 15/10/12 13:06, Dag Wanvik wrote: Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build on it. Good luck! Thanks, Dag On 14.10.2012 20:28, david myers wrote: Hi all, must apologise for being so quiet,busy with my 'real' job etc. However this doesn't mean I've been ignoring what I want to do. I've been reading around the code to determine the files that I am going to require to initiate this little modification. I think I am begining to understand how things work a little better. Firstly however I want to ensure I have well understood. The file sqlgrammar.jj is read by javacc and creates a bunch of other .java clases. SQLParserConstants.java is one of these files, it is an interface that is implemented by the SQLParser class (also created from sqlgrammar.jj). So if I can modify sqlgrammar.jj I could 'inject' into the SQLParser class a method that return the list of static final members (stored in the tokenImage string array member, so in fact I could just return this member). My problems currently now revolve around the following. DAG has created a method that reads the sqlgrammar.jj file directly and creates the Keywords.java file. I can't seem to find this in my source tree, is this related to needing the specific DERBY-3256 checkout. I've tried using the SVN plugin for eclipse to find this checkout, but so far to no success. Any chance someone could walk me through doing this from the CLI (ubuntu 12.04). Should I mention at this point I'm rather new to SVN (actually version control in general!), did I say previously this is my first multi person project? sorry I digress I've located the files that I think I'm going to need. EmbededDataBaseMetaData[40].java and TestDBMetaData.java where I will put my < getDerbyKeyWords() > method and test. Rik thanks for waving your hands about the compile time tool. I think this may be what I need to implement my < getTokenImage() > so any pointers would be great. Once I've got the method working locally I guess I'll need to create a bug or something, that points to the various JIRA issues, and then solve it but for me that comes a little later. For now I want to write the method into my local tree only. Then recompile, test ... blah blah blah... Is this an acceptable way in which to proceed? Any suggestions advice are welcome. Thanks in advance. David. On 10/09/12 01:23, Dag Wanvik wrote: Hi, I added an experimental patch to DERBY-3256 which seems to do (part of) the job. It uses the actual lists from sqlgrammar.jj to generate Keywords.java at compile time, and wraps those in a table function. There is no metadata function in it but it could be used to provide
Re: Derby Key Words
Hello All, As always first off I should appologise for being quiet, i've been reading through the code and I think I start to understand things that were not very clear to me previously. My current situation is now the following I've got myself well sorted out. Eclipse now has a 'handle' on the svn repo (copied locally of course), and I've been able to grab Dag's patch and it to this, all from inside eclipse (which I'm pleased worked out). Dag's code is interesting, very cool. I feel my next steps are as follows... I need to write a new class (or extend an existing one). I guess something in DB MetaData. But what and where? Then I need to test it. Where should I put my test class (or again should I add the test to a class that already exists. I feel that I am now ready to implement this. I just need some pointers of how to run my individual test case (I make the assumption that as I'm not going near anything else in the first instance I shouldn't break it (that is also why I propose a new subclass of one of the existing dbMetaData classes). I hope to become a little more active before christmass David. On 15/10/12 20:48, david myers wrote: Dag, thanks for the info. I'll have a try with that later on. One thing though, should I not be in my < derby trunk sandbox of checked out from svn[1] > before doing wget (surely I'll end up with it in the wrong place)... I'm probably showing my ignorance about wget and svn as I've not done this sort of stuff before ;) Not that it matters, I'll check it out later on David On 15/10/12 13:06, Dag Wanvik wrote: Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build on it. Good luck! Thanks, Dag On 14.10.2012 20:28, david myers wrote: Hi all, must apologise for being so quiet,busy with my 'real' job etc. However this doesn't mean I've been ignoring what I want to do. I've been reading around the code to determine the files that I am going to require to initiate this little modification. I think I am begining to understand how things work a little better. Firstly however I want to ensure I have well understood. The file sqlgrammar.jj is read by javacc and creates a bunch of other .java clases. SQLParserConstants.java is one of these files, it is an interface that is implemented by the SQLParser class (also created from sqlgrammar.jj). So if I can modify sqlgrammar.jj I could 'inject' into the SQLParser class a method that return the list of static final members (stored in the tokenImage string array member, so in fact I could just return this member). My problems currently now revolve around the following. DAG has created a method that reads the sqlgrammar.jj file directly and creates the Keywords.java file. I can't seem to find this in my source tree, is this related to needing the specific DERBY-3256 checkout. I've tried using the SVN plugin for eclipse to find this checkout, but so far to no success. Any chance someone could walk me through doing this from the CLI (ubuntu 12.04). Should I mention at this point I'm rather new to SVN (actually version control in general!), did I say previously this is my first multi person project? sorry I digress I've located the files that I think I'm going to need. EmbededDataBaseMetaData[40].java and TestDBMetaData.java where I will put my < getDerbyKeyWords() > method and test. Rik thanks for waving your hands about the compile time tool. I think this may be what I need to implement my < getTokenImage() > so any pointers would be great. Once I've got the method working locally I guess I'll need to create a bug or something, that points to the various JIRA issues, and then solve it but for me that comes a little later. For now I want to write the method into my local tree only. Then recompile, test ... blah blah blah... Is this an acceptable way in which to proceed? Any suggestions advice are welcome. Thanks in advance. David. On 10/09/12 01:23, Dag Wanvik wrote: Hi, I added an experimental patch to DERBY-3256 which seems to do (part of) the job. It uses the actual lists from sqlgrammar.jj to generate Keywords.java at compile time, and wraps those in a table function. There is no metadata function in it but it could be used to provide those functions, I think, if we decide that's the way to go. Dag On 05.09.2012 23:28, david myers wrote: On 05/09/12 06:25, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have bui
Re: Derby Key Words
Yes, sorry, you'd need to do that or use a path for the patch file, of course. Dag david myers wrote: >Dag, > >thanks for the info. I'll have a try with that later on. >One thing though, should I not be in my < derby trunk sandbox of checked >out from svn[1] > before doing wget (surely I'll end up with it in the >wrong place)... I'm probably showing my ignorance about wget and svn as >I've not done this sort of stuff before ;) > >Not that it matters, I'll check it out later on > >David > >On 15/10/12 13:06, Dag Wanvik wrote: >> Hi David, >> >> to access the patch I made, you need to download the patch attached to >> the JIRA issue, e.g. this way >> >> $ wget --no-check-certificate >> https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff >> $ cd >> $ patch -p0 < derbykeywords-1.diff >> >> Hope this works! >> >> [1] See here >> http://db.apache.org/derby/dev/derby_source.html#Development+trunk >> $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ >> >> Since my patch only did part of what you wanted, I didn't commit it yet, >> so feel free to build on it. >> Good luck! >> >> Thanks, >> Dag >> >> On 14.10.2012 20:28, david myers wrote: >>> Hi all, >>> >>> must apologise for being so quiet,busy with my 'real' job etc. However >>> this doesn't mean I've been ignoring what I want to do. >>> >>> I've been reading around the code to determine the files that I am >>> going to require to initiate this little modification. I think I am >>> begining to understand how things work a little better. Firstly >>> however I want to ensure I have well understood. >>> >>> The file sqlgrammar.jj is read by javacc and creates a bunch of other >>> .java clases. >>> >>> SQLParserConstants.java is one of these files, it is an interface that >>> is implemented by the SQLParser class (also created from sqlgrammar.jj). >>> >>> So if I can modify sqlgrammar.jj I could 'inject' into the SQLParser >>> class a method that return the list of static final members (stored in >>> the tokenImage string array member, so in fact I could just return >>> this member). >>> >>> My problems currently now revolve around the following. >>> >>> DAG has created a method that reads the sqlgrammar.jj file directly >>> and creates the Keywords.java file. I can't seem to find this in my >>> source tree, is this related to needing the specific DERBY-3256 >>> checkout. I've tried using the SVN plugin for eclipse to find this >>> checkout, but so far to no success. Any chance someone could walk me >>> through doing this from the CLI (ubuntu 12.04). >>> >>> Should I mention at this point I'm rather new to SVN (actually version >>> control in general!), did I say previously this is my first multi >>> person project? >>> sorry I digress >>> >>> I've located the files that I think I'm going to need. >>> EmbededDataBaseMetaData[40].java and TestDBMetaData.java where I will >>> put my < getDerbyKeyWords() > method and test. >>> >>> Rik thanks for waving your hands about the compile time tool. I think >>> this may be what I need to implement my < getTokenImage() > so any >>> pointers would be great. >>> >>> Once I've got the method working locally I guess I'll need to create a >>> bug or something, that points to the various JIRA issues, and then >>> solve it but for me that comes a little later. For now I want to >>> write the method into my local tree only. Then recompile, test ... >>> blah blah blah... >>> >>> Is this an acceptable way in which to proceed? Any suggestions advice >>> are welcome. >>> >>> Thanks in advance. >>> >>> David. >>> >>> >>> >>> >>> On 10/09/12 01:23, Dag Wanvik wrote: Hi, I added an experimental patch to DERBY-3256 which seems to do (part of) the job. It uses the actual lists from sqlgrammar.jj to generate Keywords.java at compile time, and wraps those in a table function. There is no metadata function in it but it could be used to provide those functions, I think, if we decide that's the way to go. Dag On 05.09.2012 23:28, david myers wrote: > On 05/09/12 06:25, Katherine Marsden wrote: >> On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! >>> It's possible that Eclipse doesn't grok '.jj' files in its normal >>> configuration. >>> >>> The file should be in your source tree as: >>> >>> ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj >>> >>> There is a compiler-generation tool that processes this file during >>> the Ant build of Derby and generates Java source from the grammar. >>> >> The tool that generates the java code for the parser from >> sqlgrammar.jj
Re: Derby Key Words
Dag, thanks for the info. I'll have a try with that later on. One thing though, should I not be in my < derby trunk sandbox of checked out from svn[1] > before doing wget (surely I'll end up with it in the wrong place)... I'm probably showing my ignorance about wget and svn as I've not done this sort of stuff before ;) Not that it matters, I'll check it out later on David On 15/10/12 13:06, Dag Wanvik wrote: Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build on it. Good luck! Thanks, Dag On 14.10.2012 20:28, david myers wrote: Hi all, must apologise for being so quiet,busy with my 'real' job etc. However this doesn't mean I've been ignoring what I want to do. I've been reading around the code to determine the files that I am going to require to initiate this little modification. I think I am begining to understand how things work a little better. Firstly however I want to ensure I have well understood. The file sqlgrammar.jj is read by javacc and creates a bunch of other .java clases. SQLParserConstants.java is one of these files, it is an interface that is implemented by the SQLParser class (also created from sqlgrammar.jj). So if I can modify sqlgrammar.jj I could 'inject' into the SQLParser class a method that return the list of static final members (stored in the tokenImage string array member, so in fact I could just return this member). My problems currently now revolve around the following. DAG has created a method that reads the sqlgrammar.jj file directly and creates the Keywords.java file. I can't seem to find this in my source tree, is this related to needing the specific DERBY-3256 checkout. I've tried using the SVN plugin for eclipse to find this checkout, but so far to no success. Any chance someone could walk me through doing this from the CLI (ubuntu 12.04). Should I mention at this point I'm rather new to SVN (actually version control in general!), did I say previously this is my first multi person project? sorry I digress I've located the files that I think I'm going to need. EmbededDataBaseMetaData[40].java and TestDBMetaData.java where I will put my < getDerbyKeyWords() > method and test. Rik thanks for waving your hands about the compile time tool. I think this may be what I need to implement my < getTokenImage() > so any pointers would be great. Once I've got the method working locally I guess I'll need to create a bug or something, that points to the various JIRA issues, and then solve it but for me that comes a little later. For now I want to write the method into my local tree only. Then recompile, test ... blah blah blah... Is this an acceptable way in which to proceed? Any suggestions advice are welcome. Thanks in advance. David. On 10/09/12 01:23, Dag Wanvik wrote: Hi, I added an experimental patch to DERBY-3256 which seems to do (part of) the job. It uses the actual lists from sqlgrammar.jj to generate Keywords.java at compile time, and wraps those in a table function. There is no metadata function in it but it could be used to provide those functions, I think, if we decide that's the way to go. Dag On 05.09.2012 23:28, david myers wrote: On 05/09/12 06:25, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey Hello again all, Sorry I hate to pollute the thread, but I want to ensure that I do things 'the apache way' and not just my way! So I need a little advice. First some info. The file as mentioned by Bryan has been located. Thanks to Kathey's pointer I have found the actual java file that is going to be of interest to me. The files that I think are going to be inform
Re: Derby Key Words
Hi David, to access the patch I made, you need to download the patch attached to the JIRA issue, e.g. this way $ wget --no-check-certificate https://issues.apache.org/jira/secure/attachment/12544418/derbykeywords-1.diff $ cd $ patch -p0 < derbykeywords-1.diff Hope this works! [1] See here http://db.apache.org/derby/dev/derby_source.html#Development+trunk $ svn checkout https://svn.apache.org/repos/asf/db/derby/code/trunk/ Since my patch only did part of what you wanted, I didn't commit it yet, so feel free to build on it. Good luck! Thanks, Dag On 14.10.2012 20:28, david myers wrote: > Hi all, > > must apologise for being so quiet,busy with my 'real' job etc. However > this doesn't mean I've been ignoring what I want to do. > > I've been reading around the code to determine the files that I am > going to require to initiate this little modification. I think I am > begining to understand how things work a little better. Firstly > however I want to ensure I have well understood. > > The file sqlgrammar.jj is read by javacc and creates a bunch of other > .java clases. > > SQLParserConstants.java is one of these files, it is an interface that > is implemented by the SQLParser class (also created from sqlgrammar.jj). > > So if I can modify sqlgrammar.jj I could 'inject' into the SQLParser > class a method that return the list of static final members (stored in > the tokenImage string array member, so in fact I could just return > this member). > > My problems currently now revolve around the following. > > DAG has created a method that reads the sqlgrammar.jj file directly > and creates the Keywords.java file. I can't seem to find this in my > source tree, is this related to needing the specific DERBY-3256 > checkout. I've tried using the SVN plugin for eclipse to find this > checkout, but so far to no success. Any chance someone could walk me > through doing this from the CLI (ubuntu 12.04). > > Should I mention at this point I'm rather new to SVN (actually version > control in general!), did I say previously this is my first multi > person project? > sorry I digress > > I've located the files that I think I'm going to need. > EmbededDataBaseMetaData[40].java and TestDBMetaData.java where I will > put my < getDerbyKeyWords() > method and test. > > Rik thanks for waving your hands about the compile time tool. I think > this may be what I need to implement my < getTokenImage() > so any > pointers would be great. > > Once I've got the method working locally I guess I'll need to create a > bug or something, that points to the various JIRA issues, and then > solve it but for me that comes a little later. For now I want to > write the method into my local tree only. Then recompile, test ... > blah blah blah... > > Is this an acceptable way in which to proceed? Any suggestions advice > are welcome. > > Thanks in advance. > > David. > > > > > On 10/09/12 01:23, Dag Wanvik wrote: >> Hi, >> >> I added an experimental patch to DERBY-3256 which seems to do (part >> of) the job. It uses the actual lists from sqlgrammar.jj to generate >> Keywords.java at compile time, and wraps those in a table function. >> There is no metadata function in it but it could be used to provide >> those functions, I think, if we decide that's the way to go. >> >> Dag >> >> On 05.09.2012 23:28, david myers wrote: >>> On 05/09/12 06:25, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: >> I've had success in pointing Eclipse to the source code that I >> have built. I have had a nice look around for the file that I am >> interested in, which from the previously mentioned JIRA issues >> etc suggest a file called sqlgrammar.jj >> only problem is I can't find it! > > It's possible that Eclipse doesn't grok '.jj' files in its normal > configuration. > > The file should be in your source tree as: > > ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj > > There is a compiler-generation tool that processes this file during > the Ant build of Derby and generates Java source from the grammar. > The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey >>> Hello again all, >>> >>> Sorry I hate to pollute the thread, but I want to ensure that I do >>> things 'the apache way' and not just my way! >>> So I need a little advice. >>> First some info. >>> >>> The file as mentioned by Bryan has been located. Thanks to Kathey's >>> pointer I have found the actual java file that is going to be of >>> interest to me. >>> >>> The files that I think are going to be informative are: >>> SQLParserConstants.java ~ this is the interface >>> SQLParser ~ a class file that impleme
Re: Derby Key Words
Hi all, must apologise for being so quiet,busy with my 'real' job etc. However this doesn't mean I've been ignoring what I want to do. I've been reading around the code to determine the files that I am going to require to initiate this little modification. I think I am begining to understand how things work a little better. Firstly however I want to ensure I have well understood. The file sqlgrammar.jj is read by javacc and creates a bunch of other .java clases. SQLParserConstants.java is one of these files, it is an interface that is implemented by the SQLParser class (also created from sqlgrammar.jj). So if I can modify sqlgrammar.jj I could 'inject' into the SQLParser class a method that return the list of static final members (stored in the tokenImage string array member, so in fact I could just return this member). My problems currently now revolve around the following. DAG has created a method that reads the sqlgrammar.jj file directly and creates the Keywords.java file. I can't seem to find this in my source tree, is this related to needing the specific DERBY-3256 checkout. I've tried using the SVN plugin for eclipse to find this checkout, but so far to no success. Any chance someone could walk me through doing this from the CLI (ubuntu 12.04). Should I mention at this point I'm rather new to SVN (actually version control in general!), did I say previously this is my first multi person project? sorry I digress I've located the files that I think I'm going to need. EmbededDataBaseMetaData[40].java and TestDBMetaData.java where I will put my < getDerbyKeyWords() > method and test. Rik thanks for waving your hands about the compile time tool. I think this may be what I need to implement my < getTokenImage() > so any pointers would be great. Once I've got the method working locally I guess I'll need to create a bug or something, that points to the various JIRA issues, and then solve it but for me that comes a little later. For now I want to write the method into my local tree only. Then recompile, test ... blah blah blah... Is this an acceptable way in which to proceed? Any suggestions advice are welcome. Thanks in advance. David. On 10/09/12 01:23, Dag Wanvik wrote: Hi, I added an experimental patch to DERBY-3256 which seems to do (part of) the job. It uses the actual lists from sqlgrammar.jj to generate Keywords.java at compile time, and wraps those in a table function. There is no metadata function in it but it could be used to provide those functions, I think, if we decide that's the way to go. Dag On 05.09.2012 23:28, david myers wrote: On 05/09/12 06:25, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey Hello again all, Sorry I hate to pollute the thread, but I want to ensure that I do things 'the apache way' and not just my way! So I need a little advice. First some info. The file as mentioned by Bryan has been located. Thanks to Kathey's pointer I have found the actual java file that is going to be of interest to me. The files that I think are going to be informative are: SQLParserConstants.java ~ this is the interface SQLParser ~ a class file that implements this interface is This file is found in Source Folder: generated/java/ package: org.apache.derby.impl.sql.compile So now it is a question of testing out my idea ! The interface has no methods, only static final "int" members. This leads me to a potential solution as follows: Create a method that has an inner class that implements the SQLParserConstants.java interface This inner class has a single method that grabs all of the members and inserts them into a arrayList - for "equal time access" to the values. We can then access this arrayList and compare a passed in value to determine if the value exists in the list. My first idea is to test the method within a test class, a good candidate for its location is the following I think. TestDbMetaData located in source folder java/test package ~ org.apache.der
Re: Derby Key Words
Hi, I added an experimental patch to DERBY-3256 which seems to do (part of) the job. It uses the actual lists from sqlgrammar.jj to generate Keywords.java at compile time, and wraps those in a table function. There is no metadata function in it but it could be used to provide those functions, I think, if we decide that's the way to go. Dag On 05.09.2012 23:28, david myers wrote: On 05/09/12 06:25, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey Hello again all, Sorry I hate to pollute the thread, but I want to ensure that I do things 'the apache way' and not just my way! So I need a little advice. First some info. The file as mentioned by Bryan has been located. Thanks to Kathey's pointer I have found the actual java file that is going to be of interest to me. The files that I think are going to be informative are: SQLParserConstants.java ~ this is the interface SQLParser ~ a class file that implements this interface is This file is found in Source Folder: generated/java/ package: org.apache.derby.impl.sql.compile So now it is a question of testing out my idea ! The interface has no methods, only static final "int" members. This leads me to a potential solution as follows: Create a method that has an inner class that implements the SQLParserConstants.java interface This inner class has a single method that grabs all of the members and inserts them into a arrayList - for "equal time access" to the values. We can then access this arrayList and compare a passed in value to determine if the value exists in the list. My first idea is to test the method within a test class, a good candidate for its location is the following I think. TestDbMetaData located in source folder java/test package ~ org.apache.derbyTesting.functionTests.tests.jdbc4 Or alternatively I can create a separate test class to test the idea (which would probably be my preference to start with, then once it works as I would like it to everything can be copied into a more sensible location. Once I have the method functioning I could add it into a specific package (probably databaseMetaData), then test it in situ. Does the above seem like a valid process? I have a few queries about the test suite. I have been using Junit4 from within eclipse, and I'm aware that you use Junit 3.8. I notice that with your test files you don't use annotations for the methods. With the exception of the setup and tear down methods, how do you control the flow through the test? In Junit 4 you place a <@Test> token on the line before the method if you want the method to run, along with annotations like <@before> if there are any that need to come before other stuff. Also my test classes have only really had 1 method I want to run at a time, so I just comment out the <@Test> annotation to effectively turn off the method. Please add your comments. David
Re: Derby Key Words
Hi David, Some comments inline... ... re your comment of String constants in an array. I found the same thing in the interface. It has just occured to me that, as we use a database, maybe a resultSet may be the thing to return , I've come full circle to understanding your original idea of a 'Table Function'. But I would still think that a function in DatabaseMetaData would be good. Along these lines... it should take a String argument that is the word being tested, and return true [word is in list] or false [word is not in the list]. I propose to call the method prototype to be something like getAllDerbyKeyWords(String wordToTest) . Hopefully as it has the word 'Derby' in it the SQL people are unlikely to ever use is in the standard! Although maybe there would be an argument to propose a new method of getAllKeywords() to the standard, that each driver vendor could implement and would have the same / similar functionality - delusions of grandeur ;) The presence of Derby in the function name sounds good to me. I agree that that should prevent the JDBC spec from encroaching on this function. I'm afraid I don't understand the usefulness of the String argument. Having a no-arg function which returns all of the words sounds good enough to me. The restriction to a single string can be accomplished by this query against the resulting table function: select keyword from table( getAllDerbyKeyWords() ) t where t.keyword = 'wordToTest' But how to turn that String Array into a Table is fairly simple (I'd declare a temp table), and then make a select on it to get a resultSet. However is there a way to skip the table step and jump straight to a result set (its a bit of a leading question, and I haven't searched for an answer anywhere yet, so unless you know off the top of your head no need to answer straight away). The table function can just be an extension of org.apache.derby.vti.StringColumnVTI. All you have to implement are 3 methods: next(), close(), and getRawColumn(). Hope this helps, -Rick David
Re: Derby Key Words
On 05/09/12 23:29, Rick Hillegas wrote: On 9/5/12 7:02 AM, David Myers wrote: Hello all, Bryan, thanks for the info (although I was using 'locate' on my ubuntu box, and it still didn't find it ?) I'll have a look tonight. Kathey, I don't intend to modify the file, just grab the contents, if possible an without using a file Input Stream Hi David, Note that the keywords appear in an array of String constants that are generated by javacc into trunk/generated/java/org/apache/derby/impl/sql/compile/SQLParserConstants.java. That may be a more straightforward way to get your hands on the keywords but it may require understanding more about how the generated grammar behaves and it may involve building a compile time tool which scrapes the keywords out of SQLParserConstants. I am waving my hands, of course. Hope this helps, -Rick I also noted that the DatabaseMetaData.getSQLkeywords() actually has a hard coded list of words! I have no problem with that in the first instance, but I'm sure that the engine doesn' check every word with a call to this method before throwing an error! I need to keep looking...I'll keep you all posted. D On Wed, Sep 5, 2012 at 6:25 AM, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey Rick, re your comment of String constants in an array. I found the same thing in the interface. It has just occured to me that, as we use a database, maybe a resultSet may be the thing to return , I've come full circle to understanding your original idea of a 'Table Function'. But I would still think that a function in DatabaseMetaData would be good. Along these lines... it should take a String argument that is the word being tested, and return true [word is in list] or false [word is not in the list]. I propose to call the method prototype to be something like getAllDerbyKeyWords(String wordToTest) . Hopefully as it has the word 'Derby' in it the SQL people are unlikely to ever use is in the standard! Although maybe there would be an argument to propose a new method of getAllKeywords() to the standard, that each driver vendor could implement and would have the same / similar functionality - delusions of grandeur ;) But how to turn that String Array into a Table is fairly simple (I'd declare a temp table), and then make a select on it to get a resultSet. However is there a way to skip the table step and jump straight to a result set (its a bit of a leading question, and I haven't searched for an answer anywhere yet, so unless you know off the top of your head no need to answer straight away). David
Re: Derby Key Words
On 9/5/12 7:02 AM, David Myers wrote: Hello all, Bryan, thanks for the info (although I was using 'locate' on my ubuntu box, and it still didn't find it ?) I'll have a look tonight. Kathey, I don't intend to modify the file, just grab the contents, if possible an without using a file Input Stream Hi David, Note that the keywords appear in an array of String constants that are generated by javacc into trunk/generated/java/org/apache/derby/impl/sql/compile/SQLParserConstants.java. That may be a more straightforward way to get your hands on the keywords but it may require understanding more about how the generated grammar behaves and it may involve building a compile time tool which scrapes the keywords out of SQLParserConstants. I am waving my hands, of course. Hope this helps, -Rick I also noted that the DatabaseMetaData.getSQLkeywords() actually has a hard coded list of words! I have no problem with that in the first instance, but I'm sure that the engine doesn' check every word with a call to this method before throwing an error! I need to keep looking...I'll keep you all posted. D On Wed, Sep 5, 2012 at 6:25 AM, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey
Re: Derby Key Words
On 05/09/12 06:25, Katherine Marsden wrote: On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey Hello again all, Sorry I hate to pollute the thread, but I want to ensure that I do things 'the apache way' and not just my way! So I need a little advice. First some info. The file as mentioned by Bryan has been located. Thanks to Kathey's pointer I have found the actual java file that is going to be of interest to me. The files that I think are going to be informative are: SQLParserConstants.java ~ this is the interface SQLParser ~ a class file that implements this interface is This file is found in Source Folder: generated/java/ package: org.apache.derby.impl.sql.compile So now it is a question of testing out my idea ! The interface has no methods, only static final "int" members. This leads me to a potential solution as follows: Create a method that has an inner class that implements the SQLParserConstants.java interface This inner class has a single method that grabs all of the members and inserts them into a arrayList - for "equal time access" to the values. We can then access this arrayList and compare a passed in value to determine if the value exists in the list. My first idea is to test the method within a test class, a good candidate for its location is the following I think. TestDbMetaData located in source folder java/test package ~ org.apache.derbyTesting.functionTests.tests.jdbc4 Or alternatively I can create a separate test class to test the idea (which would probably be my preference to start with, then once it works as I would like it to everything can be copied into a more sensible location. Once I have the method functioning I could add it into a specific package (probably databaseMetaData), then test it in situ. Does the above seem like a valid process? I have a few queries about the test suite. I have been using Junit4 from within eclipse, and I'm aware that you use Junit 3.8. I notice that with your test files you don't use annotations for the methods. With the exception of the setup and tear down methods, how do you control the flow through the test? In Junit 4 you place a <@Test> token on the line before the method if you want the method to run, along with annotations like <@before> if there are any that need to come before other stuff. Also my test classes have only really had 1 method I want to run at a time, so I just comment out the <@Test> annotation to effectively turn off the method. Please add your comments. David
Re: Derby Key Words
On Wed, Sep 5, 2012 at 4:02 AM, David Myers wrote: > Hello all, > > Bryan, thanks for the info (although I was using 'locate' on my ubuntu > box, and it still didn't find it ?) I'll have a look tonight. > > Kathey, I don't intend to modify the file, just grab the contents, if > possible an without using a file Input Stream > > I also noted that the DatabaseMetaData.getSQLkeywords() actually has a > hard coded list of words! I have no problem with that in the first > instance, but I'm sure that the engine doesn' check every word with a > call to this method before throwing an error! > > I need to keep looking...I'll keep you all posted. > > D > > On Wed, Sep 5, 2012 at 6:25 AM, Katherine Marsden > wrote: >> On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! >>> >>> >>> It's possible that Eclipse doesn't grok '.jj' files in its normal >>> configuration. >>> >>> The file should be in your source tree as: >>> >>> ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj >>> >>> There is a compiler-generation tool that processes this file during >>> the Ant build of Derby and generates Java source from the grammar. >>> >> The tool that generates the java code for the parser from sqlgrammar.jj is >> Javacc. The generated code is in >> generated/java/org/apache/derby/impl/sql/compile which is fun to look at but >> shouldn't be changed. >> I think there may be an exclipse plugin for javacc but I have never used >> one. >> >> Kathey >> >> Hi, In eclipse (using the default shortcut keys), I can find sqlgrammar.jj when I type the name in the screen that pops up on Ctrl-Shift-R...No special configuration is needed. I also wanted to mention, that in some cases, Derby allows SQL 2003 reserved keywords as identifiers for backward compatibility - see for instance the comments from DERBY-5254 (https://issues.apache.org/jira/browse/DERBY-5254). Other DBMSs may be more strict... Myrna
Re: Derby Key Words
Hello all, Bryan, thanks for the info (although I was using 'locate' on my ubuntu box, and it still didn't find it ?) I'll have a look tonight. Kathey, I don't intend to modify the file, just grab the contents, if possible an without using a file Input Stream I also noted that the DatabaseMetaData.getSQLkeywords() actually has a hard coded list of words! I have no problem with that in the first instance, but I'm sure that the engine doesn' check every word with a call to this method before throwing an error! I need to keep looking...I'll keep you all posted. D On Wed, Sep 5, 2012 at 6:25 AM, Katherine Marsden wrote: > On 9/4/2012 7:44 PM, Bryan Pendleton wrote: >>> >>> I've had success in pointing Eclipse to the source code that I have >>> built. I have had a nice look around for the file that I am >>> interested in, which from the previously mentioned JIRA issues etc >>> suggest a file called sqlgrammar.jj >>> only problem is I can't find it! >> >> >> It's possible that Eclipse doesn't grok '.jj' files in its normal >> configuration. >> >> The file should be in your source tree as: >> >> ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj >> >> There is a compiler-generation tool that processes this file during >> the Ant build of Derby and generates Java source from the grammar. >> > The tool that generates the java code for the parser from sqlgrammar.jj is > Javacc. The generated code is in > generated/java/org/apache/derby/impl/sql/compile which is fun to look at but > shouldn't be changed. > I think there may be an exclipse plugin for javacc but I have never used > one. > > Kathey > >
Re: Derby Key Words
On 9/4/2012 7:44 PM, Bryan Pendleton wrote: I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. The tool that generates the java code for the parser from sqlgrammar.jj is Javacc. The generated code is in generated/java/org/apache/derby/impl/sql/compile which is fun to look at but shouldn't be changed. I think there may be an exclipse plugin for javacc but I have never used one. Kathey
Re: Derby Key Words
I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! It's possible that Eclipse doesn't grok '.jj' files in its normal configuration. The file should be in your source tree as: ./java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj There is a compiler-generation tool that processes this file during the Ant build of Derby and generates Java source from the grammar. thanks, bryan
Re: Derby Key Words
On 31/08/12 15:00, Rick Hillegas wrote: On 8/30/12 2:56 PM, David Myers wrote: ... I think your solution of a table function should work well enough. > From the few table functions I have used, I notice that I was also able to get the same info from a join of 2 system tables. Is this how they generally work (I call the function and an SQL command runs in the background)? Hi David, What you just described sounds more like a view than a table function to me. Table functions are useful for examining data that comes from external databases or from non-relational sources. Turning a list into a table sounds like a good use of table functions to me. Also would it not make some sense to have a method within the databaseMetaData object thqt would return either the list of values (after calling the new function) or take a string argument and return true (the argument is equivalent to a key word) or false. This is an interesting approach and might be cheaper to implement. I think we would only need to add the new method to Derby's embedded implementation of DatabaseMetaData. We would need to make sure that the new method had a name which the JDBC expert group would never want to use. The advantages of a table function (can be used in queries, can be used across the network by JDBC client apps) could be had by wrapping the Derby DatabaseMetaData in something like the DBMDWrapper class attached to https://issues.apache.org/jira/browse/DERBY-3973. Thanks, -Rick Hello again all. Things are progressing sort of ;) I've had success in pointing Eclipse to the source code that I have built. I have had a nice look around for the file that I am interested in, which from the previously mentioned JIRA issues etc suggest a file called sqlgrammar.jj only problem is I can't find it! This file holds the 2 methods, that I am interested in. It seems to me that the data must already be locked away somewhere in derby, but I haven't yet found where. I haven't even found a databasemetadata.java file... yet ? I guess I'm missing something somewhere... David
Re: Derby Key Words
On 8/30/12 2:56 PM, David Myers wrote: ... I think your solution of a table function should work well enough. > From the few table functions I have used, I notice that I was also able to get the same info from a join of 2 system tables. Is this how they generally work (I call the function and an SQL command runs in the background)? Hi David, What you just described sounds more like a view than a table function to me. Table functions are useful for examining data that comes from external databases or from non-relational sources. Turning a list into a table sounds like a good use of table functions to me. Also would it not make some sense to have a method within the databaseMetaData object thqt would return either the list of values (after calling the new function) or take a string argument and return true (the argument is equivalent to a key word) or false. This is an interesting approach and might be cheaper to implement. I think we would only need to add the new method to Derby's embedded implementation of DatabaseMetaData. We would need to make sure that the new method had a name which the JDBC expert group would never want to use. The advantages of a table function (can be used in queries, can be used across the network by JDBC client apps) could be had by wrapping the Derby DatabaseMetaData in something like the DBMDWrapper class attached to https://issues.apache.org/jira/browse/DERBY-3973. Thanks, -Rick
Re: Derby Key Words
On 8/30/2012 2:56 PM, David Myers wrote: e a quick look at the file you mention tomorrow. On my original post I mentioned I use eclipse, are there any instructions on how to get eclipse to 'see' the project, I prefer the interface for eclipse over the rather basic editor on L-Ubuntu (and I never really got the hang of emacs or vi), or is there another prefered development platform?. Hi David, Welcome! Tiago made this very nice screen cast on setting up Eclipse. off the bottom of this wiki page http://wiki.apache.org/db-derby/Tiago_Espinha direct link is: We should probably link this is an a more prominent location on the Wiki for new developers. http://www.vimeo.com/5376690
Re: Derby Key Words
Rick, I intend to do this during my spare time, even though the need first arose whilst I was at work, this for me will be a strict 'hour or so every in an evening (possibly also weekends... if I'm not busy breaking / renovating the house, taking children to sports clubs etc). I think your solution of a table function should work well enough. >From the few table functions I have used, I notice that I was also able to get the same info from a join of 2 system tables. Is this how they generally work (I call the function and an SQL command runs in the background)? Also would it not make some sense to have a method within the databaseMetaData object thqt would return either the list of values (after calling the new function) or take a string argument and return true (the argument is equivalent to a key word) or false. I guess a function somewhat like this must already exist somewhere that looks for the key words in any sql commands - and kicks out an appropriate error message. I'll have a quick look at the file you mention tomorrow. On my original post I mentioned I use eclipse, are there any instructions on how to get eclipse to 'see' the project, I prefer the interface for eclipse over the rather basic editor on L-Ubuntu (and I never really got the hang of emacs or vi), or is there another prefered development platform?. Thanks David. On Thu, Aug 30, 2012 at 6:00 PM, Rick Hillegas wrote: > On 8/30/12 8:09 AM, David Myers wrote: >> >> Hello Rik, >> >> I thought the JIRA references would help. >> >> In response to your comment. >>> >>> Is it fair to say that you want to give users a programmatic way to list >>> out >>> all of the Derby keywords, reserved and non-reserved, regardless of >>> whether >>> they are keywords in the 2003 rev of the SQL Standard? The point of >>> producing this list is to help developers and IDEs avoid using these >>> words >>> as names of tables, columns, routines, etc.. >>> >> Exactly. As I say I already have the java / sql statements to do this >> (although I have copy / pasted the values from the web page rather >> than grabbing them from the system), I just need to know where I >> should plant them, and how to test that I don't break anything in the >> process ;) ! > > Thanks, David. I think that this would be a useful addition to Derby. I > don't think it requires a new system table. In general, we are reluctant to > add new system tables if a problem can be solved with a simpler mechanism > which doesn't increase the on-disk footprint of every database. > > Instead, I would recommend supplying a diagnostic table function which > behaves as though it were declared as follows: > > create function syscs_diag.derby_keywords() > returns table( keyword varchar( 128 ), isReserved boolean ) > ... > > If you are not familiar with table functions, I recommend reading the > following sections in the user guides: > > o The CREATE FUNCTION section in the Reference Guide. > > o The "Programming Derby-style table functions" section in the Developer's > Guide. > > The actual java code which implements the table function can just extend > org.apache.derby.vti.StringColumnVTI. If you are comfortable with this > solution, I would be happy to coach you through the project. The first thing > you will need to do is sign a contributor license agreement. If you will be > doing this work on behalf of your daytime job, then you will want your > company to sign a corporate contributor license agreement. The first > agreement protects Apache and the second agreement protects you. Information > on these agreements can be found here: http://www.apache.org/licenses/#clas > > Thanks, > -Rick > >
Re: Derby Key Words
On 8/30/12 8:09 AM, David Myers wrote: Hello Rik, I thought the JIRA references would help. In response to your comment. Is it fair to say that you want to give users a programmatic way to list out all of the Derby keywords, reserved and non-reserved, regardless of whether they are keywords in the 2003 rev of the SQL Standard? The point of producing this list is to help developers and IDEs avoid using these words as names of tables, columns, routines, etc.. Exactly. As I say I already have the java / sql statements to do this (although I have copy / pasted the values from the web page rather than grabbing them from the system), I just need to know where I should plant them, and how to test that I don't break anything in the process ;) ! Thanks, David. I think that this would be a useful addition to Derby. I don't think it requires a new system table. In general, we are reluctant to add new system tables if a problem can be solved with a simpler mechanism which doesn't increase the on-disk footprint of every database. Instead, I would recommend supplying a diagnostic table function which behaves as though it were declared as follows: create function syscs_diag.derby_keywords() returns table( keyword varchar( 128 ), isReserved boolean ) ... If you are not familiar with table functions, I recommend reading the following sections in the user guides: o The CREATE FUNCTION section in the Reference Guide. o The "Programming Derby-style table functions" section in the Developer's Guide. The actual java code which implements the table function can just extend org.apache.derby.vti.StringColumnVTI. If you are comfortable with this solution, I would be happy to coach you through the project. The first thing you will need to do is sign a contributor license agreement. If you will be doing this work on behalf of your daytime job, then you will want your company to sign a corporate contributor license agreement. The first agreement protects Apache and the second agreement protects you. Information on these agreements can be found here: http://www.apache.org/licenses/#clas Thanks, -Rick
Re: Derby Key Words
Hello Rik, I thought the JIRA references would help. In response to your comment. > Is it fair to say that you want to give users a programmatic way to list out > all of the Derby keywords, reserved and non-reserved, regardless of whether > they are keywords in the 2003 rev of the SQL Standard? The point of > producing this list is to help developers and IDEs avoid using these words > as names of tables, columns, routines, etc.. > Exactly. As I say I already have the java / sql statements to do this (although I have copy / pasted the values from the web page rather than grabbing them from the system), I just need to know where I should plant them, and how to test that I don't break anything in the process ;) ! There are a few other interesting 'beginner' problems on the issues page that I may have a go at afterwards, for now I have an itch I want to scratch! David. On Thu, Aug 30, 2012 at 2:53 PM, Rick Hillegas wrote: > Hi David, > > Welcome to Derby development! A couple comments inline... > > > On 8/29/12 4:38 PM, david myers wrote: >> >> Hello all, >> >> This is my first post, and my first try at 'contributing' to a project... >> >> I've come across an issue with 'keywords' in derby. Bear in mind the >> functionality as it currently stands does exactly what I would expect (and >> as is described in the docs, as far as I can tell!). >> >> My issue relates to 2 that are currently on jira >> https://issues.apache.org/jira/browse/DERBY-2245 >> https://issues.apache.org/jira/browse/DERBY-3256 >> >> Bryan Pendleton made a comment on Derby2245 >> >> https://issues.apache.org/jira/browse/DERBY-2245?focusedCommentId=12539961&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12539961 >> >> Which encompasses my expectations exactly. > > Thanks for listing these related JIRAs. This is very helpful for framing the > discussion. > >> >> Background. >> I'm converting (albeit slowly) a number of MS Access DB's to derbyDB. I >> came across the problem that MS Access seems not to care about SQL keywords >> as names for tables or columns! >> I was unable to find a 'neat' solution to test names against all the >> keywords that are reserved by Derby (so as I could then simply prepend the >> name of the table to the field). >> I did some searching and found the following page. >> http://db.apache.org/derby/javadoc/language/sqlgrammar.html >> This has 2 functions (or I assume they are functions or static variables) >> in it: reservedKeyword and nonReservedKeyword > > Is it fair to say that you want to give users a programmatic way to list out > all of the Derby keywords, reserved and non-reserved, regardless of whether > they are keywords in the 2003 rev of the SQL Standard? The point of > producing this list is to help developers and IDEs avoid using these words > as names of tables, columns, routines, etc.. > > Thanks, > -Rick >
Re: Derby Key Words
Hi David, Welcome to Derby development! A couple comments inline... On 8/29/12 4:38 PM, david myers wrote: Hello all, This is my first post, and my first try at 'contributing' to a project... I've come across an issue with 'keywords' in derby. Bear in mind the functionality as it currently stands does exactly what I would expect (and as is described in the docs, as far as I can tell!). My issue relates to 2 that are currently on jira https://issues.apache.org/jira/browse/DERBY-2245 https://issues.apache.org/jira/browse/DERBY-3256 Bryan Pendleton made a comment on Derby2245 https://issues.apache.org/jira/browse/DERBY-2245?focusedCommentId=12539961&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12539961 Which encompasses my expectations exactly. Thanks for listing these related JIRAs. This is very helpful for framing the discussion. Background. I'm converting (albeit slowly) a number of MS Access DB's to derbyDB. I came across the problem that MS Access seems not to care about SQL keywords as names for tables or columns! I was unable to find a 'neat' solution to test names against all the keywords that are reserved by Derby (so as I could then simply prepend the name of the table to the field). I did some searching and found the following page. http://db.apache.org/derby/javadoc/language/sqlgrammar.html This has 2 functions (or I assume they are functions or static variables) in it: reservedKeyword and nonReservedKeyword Is it fair to say that you want to give users a programmatic way to list out all of the Derby keywords, reserved and non-reserved, regardless of whether they are keywords in the 2003 rev of the SQL Standard? The point of producing this list is to help developers and IDEs avoid using these words as names of tables, columns, routines, etc.. Thanks, -Rick
