Re: R: R: using ASTParser and TreeWalker

Rick Hillegas Wed, 11 Nov 2009 13:51:35 -0800

Hi Flavio,

The ASTParser relies on tree-printing logic in the AST nodes while theXmlTreeWalker relies on tree walking logic in the nodes. It turns outthat the tree printing logic is more capable than the tree walking logicfor the InsertNode. I have attached a patch to derby-791 which improvesthe tree walking logic for InsertNodes. I am attaching the XmlTreeWalkeroutput for your query when I apply this patch. This makes theinformation you need visible.

If the tests run cleanly for me, I can apply this patch to the trunk.That, however, won't help you if you're running against 10.5.3.0. To getthe benefits of this change, you will need to build the Derby trunk andrun your experiments against the trunk.


Regards,
-Rick

Flavio Palumbo wrote:

Hi Rick,
i'm  bit confused ;-)

I've seen the posts at the link you provided and they are quite clear.

Now i'm trying to discover tables and fields from an insert statement (very
simple : "insert into eurofile.tdepro (dpcdst, dpcdme, dpcdpr) values(?, ?,
?)"):

If i run ASTParser i get under the node
org.apache.derby.impl.sql.compile.TableName
the table name followed by a
org.apache.derby.impl.sql.compile.ResultColumnList node and 3
org.apache.derby.impl.sql.compile.ResultColumn nodes each one with its
column name.

If i run XmlTreeWalker (better in my mind because i can control the loop
over the nodes) i cannot neither org.apache.derby.impl.sql.compile.TableName
node nor the 3 org.apache.derby.impl.sql.compile.ResultColumn nodes.

Can you please give me a hint ?

Thanks a lot.

Flavio

-----Messaggio originale-----
Da: [email protected] [mailto:[email protected]]
Inviato: lunedì 9 novembre 2009 19.16
A: Derby Discussion
Oggetto: Re: R: using ASTParser and TreeWalker


Hi Flavio,

There is some commentary on the following issue, which might help you
make some sense of the trees:
https://issues.apache.org/jira/browse/DERBY-3946

These trees are not part of Derby's public API so there's not a lot of
documentation on them other than the Derby engine's javadoc. You may
have to ask a couple more questions before you touch bottom.

Hope this helps,
-Rick



flavio palumbo wrote:

Hi Rick,
renaming junit did it ;-))
I've been able to run ASTParser and XmlTreeWalker and I've seen

that they

give a very big quantity of infos ; where can I find some docs

to discover

how to point what I need ? In particular I'm searching for

tables and fields

names included in the statement, which is the quickest way to

find them ?

Thanks a lot
Flavio

-----Original Message-----
From: Rick Hillegas <[email protected]>
To: Derby Discussion <[email protected]>
Date: Fri, 06 Nov 2009 14:01:50 -0800
Subject: Re: R: using ASTParser and TreeWalker

flavio palumbo wrote:

Hi Rich,
I've just tried to build Derby (now I'm at home with nb 6.5.1 on win

vista)

and got these messages :
.....
.....
Building zip: D:\derby2009\trunk\java\demo\toursdb\toursdb.jar
Finished building toursdb
copyfiles:
Copying 13 files to

D:\derby2009\trunk\classes\org\apache\derbyTesting\functionTests\tests\
demo

toursdb:
demo:
testing:
testing:
junitcomponents:
Compiling 1 source file to D:\derby2009\trunk\classes

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\SecurityM
anagerSetup.java:30:

package junit.extensions does not exist
import junit.extensions.TestSetup;

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\SecurityM
anagerSetup.java:31:

package junit.framework does not exist
import junit.framework.Test;

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\SecurityM
anagerSetup.java:32:

package junit.framework does not exist
import junit.framework.TestSuite;

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\SecurityM
anagerSetup.java:39:

cannot find symbol
symbol: class TestSetup
public final class SecurityManagerSetup extends TestSetup {

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\SecurityM
anagerSetup.java:73:

cannot find symbol
symbol  : class Test
location: class org.apache.derbyTesting.junit.SecurityManagerSetup
        public SecurityManagerSetup(Test test, String policyResource)
                                    ^

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\SecurityM
anagerSetup.java:94:

cannot find symbol
symbol  : class Test
location: class org.apache.derbyTesting.junit.SecurityManagerSetup
        public static Test noSecurityManager(Test test)
                                             ^

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\SecurityM
anagerSetup.java:94:

cannot find symbol
symbol  : class Test
location: class org.apache.derbyTesting.junit.SecurityManagerSetup
        public static Test noSecurityManager(Test test)
.....
.....

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\JDBC.java
:673:

cannot find symbol
symbol  : variable Assert
location: class org.apache.derbyTesting.junit.JDBC
        Assert.assertEquals("Unexpected column count:",
100 errors
D:\derby2009\trunk\build.xml:499: The following error occurred while
executing this line:
D:\derby2009\trunk\java\testing\build.xml:59: The following error

occurred

while executing this line:

D:\derby2009\trunk\java\testing\org\apache\derbyTesting\junit\build.xml
:74:

Compile failed; see the compiler error output for details.
BUILD FAILED (total time: 54 seconds)
The previous time I hadn't read carefully the building.html doc, but

now

I've put ant.jar and junit-3.8.2.jar into my

D:\derby2009\trunk\tools\java

directory.
It seems that nothing has changed :-((

Hi Flavio,

Have you tried building Derby outside Netbeans? That might help narrow
down the problem. One thing to note: the JUnit jarball should be called
junit.jar, not junit-3.8.2.jar. Try renaming the JUnit jarball and see
if that helps.

Regards,
-Rick

Thanks a lot
Flavio


-----Original Message-----
From: Rick Hillegas <[email protected]>
To: Derby Discussion <[email protected]>
Date: Fri, 06 Nov 2009 08:35:38 -0800
Subject: Re: R: using ASTParser and TreeWalker

Hi Flavio,

A couple more comments inline...

Flavio Palumbo wrote:

Hi Rich,

thanks for your super quick answer ;-))

Please tell us more about the problems you are having compiling

the

Derby trunk. We want to make the build easier.

I tried to build Derby following the guide at
http://wiki.apache.org/db-derby/BuildingDerby ; I'm currently using

NetBeans

6.5 with jdk 1.6.0 ; when I try to build the project I get a lot of
exceptions due mainly to the fact that I'm not able to add jUnit

jars

to the

project.
I'm bit hurry now, I'll try to generate again the project this week

end to

be more accurate.

Thanks for giving us this feedback. If you configure the ant build
target to be "buildsource" rather than "all", then you shouldn't

need

the junit jars. For more information, see the BUILDING.html file in

the

root directory of the Derby trunk.

Yes, you should be able to compile these classes against the

official

debug jars for 10.5.3.0.

what you mean with "debug jars" ?

If you go to the 10.5.3 download page (
http://db.apache.org/derby/releases/release-10.5.3.0.cgi ), you'll

see

a
number of downloadable distributions. Two of them have the word

"debug"

in their names. The jars in those two distributions have extra

tracing

logic built into them. The TreeWalker uses that extra tracing logic.

Hope this helps,
-Rick

Thanks a lot.

Flavio

-----Messaggio originale-----
Da: [email protected] [mailto:[email protected]]
Inviato: venerdì 6 novembre 2009 15.45
A: Derby Discussion
Oggetto: Re: using ASTParser and TreeWalker


Hi Flavio,

Flavio Palumbo wrote:

Hi all,

I tried test ASTParser and TreeWalker to try to parse SQL

strings and the

first works fine while I cannot compile the second cause I

receive

this

compile time error :
TreeWalker is not abstract and does not override abstract method
visitChildrenFirst(org.apache.derby.iapi.sql.compile.Visitable)

in

org.apache.derby.iapi.sql.compile.Visitor

is there a newer version I can use or can somebody tell me how

to fix this

problem ?

The visitChildrenFirst() method was added to the Visitor interface
recently. I have attached  new versions of TreeWalker,

XmlTreeWalker,

and XmlTreePrinter to DERBY-3946, DERBY-791, and DERBY-4415
respectively. The new versions implement the new method and should
compile against the current Derby trunk. If this doesn't fix your
problem, let me know and I'll look into this more.

also I found the post "Rick Hillegas added a comment -

25/Aug/09 04:00 PM"

on https://issues.apache.org/jira/browse/DERBY-3946 :

I have successfully compiled and run this ASTParser against the

debug

derby.jar that is part of the 10.5.3.0 distribution

it means that I can compile and run ASTParser and TreeWalker

against the

normal distribution of Derby ?

Yes, you should be able to compile these classes against the

official

debug jars for 10.5.3.0.

this would be great cause I fight a lot to compile Derby from

svn.

Please tell us more about the problems you are having compiling

the

Derby trunk. We want to make the build easier.

Thanks,
-Rick

Any hint would be appreciated.

Flavio


-----------------------------------------------------------
Il presente messaggio non costituisce un impegno contrattuale

tra SILMA S.r.l. ed il destinatario.

Le opinioni ivi espresse sono quelle dell'autore.
SILMA S.r.l. non assume alcuna responsabilita riguardo al

contenuto del presente messaggio.

Il messaggio è destinato esclusivamente al destinatario.
Il contenuto e gli allegati sono da considerarsi di natura

confidenziale

Nel caso abbiate ricevuto il presente messaggio per errore

siete pregati di comunicarlo

alla casella [email protected].

-----------------------------------------------------------
Il presente messaggio non costituisce un impegno contrattuale tra

SILMA S.r.l. ed il destinatario.

Le opinioni ivi espresse sono quelle dell'autore.
SILMA S.r.l. non assume alcuna responsabilita riguardo al contenuto

del presente messaggio.

Il messaggio è destinato esclusivamente al destinatario.
Il contenuto e gli allegati sono da considerarsi di natura

confidenziale

Nel caso abbiate ricevuto il presente messaggio per errore siete

pregati di comunicarlo

alla casella [email protected].

-----------------------------------------------------------
Il presente messaggio non costituisce un impegno contrattuale tra

SILMA S.r.l. ed il destinatario.

Le opinioni ivi espresse sono quelle dell'autore.
SILMA S.r.l. non assume alcuna responsabilita riguardo al contenuto

del presente messaggio.

Il messaggio è destinato esclusivamente al destinatario.
Il contenuto e gli allegati sono da considerarsi di natura

confidenziale

Nel caso abbiate ricevuto il presente messaggio per errore siete

pregati di comunicarlo

alla casella [email protected].

-----------------------------------------------------------
Il presente messaggio non costituisce un impegno contrattuale

tra SILMA S.r.l. ed il destinatario.

Le opinioni ivi espresse sono quelle dell'autore.
SILMA S.r.l. non assume alcuna responsabilita riguardo al

contenuto del presente messaggio.

Il messaggio è destinato esclusivamente al destinatario.
Il contenuto e gli allegati sono da considerarsi di natura confidenziale

Nel caso abbiate ricevuto il presente messaggio per errore

siete pregati di comunicarlo

alla casella [email protected].



-----------------------------------------------------------
Il presente messaggio non costituisce un impegno contrattuale tra SILMA S.r.l. 
ed il destinatario.
Le opinioni ivi espresse sono quelle dell'autore.
SILMA S.r.l. non assume alcuna responsabilita riguardo al contenuto del 
presente messaggio.
Il messaggio è destinato esclusivamente al destinatario.
Il contenuto e gli allegati sono da considerarsi di natura confidenziale

Nel caso abbiate ricevuto il presente messaggio per errore siete pregati di 
comunicarlo
alla casella [email protected].

<?xml version="1.0" encoding="UTF-8"?>
<queryTree>
    <queryText>insert into eurofile.tdepro (dpcdst, dpcdme, dpcdpr) values(?, ?, ?)</queryText>
    <node type="org.apache.derby.impl.sql.compile.InsertNode">
        <contents>
            <member value="EUROFILE.TDEPRO"/>
            <member value="null"/>
            <member name="statementType" value="INSERT"/>
        </contents>
        <node type="org.apache.derby.impl.sql.compile.RowResultSetNode">
            <contents>
                <member name="orderByList" value="null"/>
                <member name="correlation Name" value="null"/>
                <member value="null"/>
                <member value="tableNumber -1"/>
                <member value="level 0"/>
                <member name="resultSetNumber" value="0"/>
                <member name="referencedTableMap" value="null"/>
                <member name="statementResultSet" value="false"/>
            </contents>
            <node type="org.apache.derby.impl.sql.compile.ResultColumnList">
                <contents>
                    <member name="indexRow" value="false"/>
                </contents>
                <node type="org.apache.derby.impl.sql.compile.ResultColumn">
                    <contents>
                        <member name="exposedName" value="null"/>
                        <member name="name" value="null"/>
                        <member name="tableName" value="null"/>
                        <member name="isDefaultColumn" value="false"/>
                        <member name="wasDefaultColumn" value="false"/>
                        <member name="isNameGenerated" value="false"/>
                        <member name="sourceTableName" value="null"/>
                        <member name="type" value="null"/>
                        <member name="columnDescriptor" value="null"/>
                        <member name="isGenerated" value="false"/>
                        <member name="isGeneratedForUnmatchedColumnInInsert" value="false"/>
                        <member name="isGroupingColumn" value="false"/>
                        <member name="isReferenced" value="false"/>
                        <member name="isRedundant" value="false"/>
                        <member name="virtualColumnId" value="1"/>
                        <member name="resultSetNumber" value="-1"/>
                        <member name="dataTypeServices" value="null"/>
                    </contents>
                    <node type="org.apache.derby.impl.sql.compile.ParameterNode">
                        <contents>
                            <member name="dataTypeServices" value="null"/>
                        </contents>
                    </node>
                </node>
                <node type="org.apache.derby.impl.sql.compile.ResultColumn">
                    <contents>
                        <member name="exposedName" value="null"/>
                        <member name="name" value="null"/>
                        <member name="tableName" value="null"/>
                        <member name="isDefaultColumn" value="false"/>
                        <member name="wasDefaultColumn" value="false"/>
                        <member name="isNameGenerated" value="false"/>
                        <member name="sourceTableName" value="null"/>
                        <member name="type" value="null"/>
                        <member name="columnDescriptor" value="null"/>
                        <member name="isGenerated" value="false"/>
                        <member name="isGeneratedForUnmatchedColumnInInsert" value="false"/>
                        <member name="isGroupingColumn" value="false"/>
                        <member name="isReferenced" value="false"/>
                        <member name="isRedundant" value="false"/>
                        <member name="virtualColumnId" value="2"/>
                        <member name="resultSetNumber" value="-1"/>
                        <member name="dataTypeServices" value="null"/>
                    </contents>
                    <node type="org.apache.derby.impl.sql.compile.ParameterNode">
                        <contents>
                            <member name="dataTypeServices" value="null"/>
                        </contents>
                    </node>
                </node>
                <node type="org.apache.derby.impl.sql.compile.ResultColumn">
                    <contents>
                        <member name="exposedName" value="null"/>
                        <member name="name" value="null"/>
                        <member name="tableName" value="null"/>
                        <member name="isDefaultColumn" value="false"/>
                        <member name="wasDefaultColumn" value="false"/>
                        <member name="isNameGenerated" value="false"/>
                        <member name="sourceTableName" value="null"/>
                        <member name="type" value="null"/>
                        <member name="columnDescriptor" value="null"/>
                        <member name="isGenerated" value="false"/>
                        <member name="isGeneratedForUnmatchedColumnInInsert" value="false"/>
                        <member name="isGroupingColumn" value="false"/>
                        <member name="isReferenced" value="false"/>
                        <member name="isRedundant" value="false"/>
                        <member name="virtualColumnId" value="3"/>
                        <member name="resultSetNumber" value="-1"/>
                        <member name="dataTypeServices" value="null"/>
                    </contents>
                    <node type="org.apache.derby.impl.sql.compile.ParameterNode">
                        <contents>
                            <member name="dataTypeServices" value="null"/>
                        </contents>
                    </node>
                </node>
            </node>
        </node>
        <node type="org.apache.derby.impl.sql.compile.TableName">
            <contents>
                <member value="EUROFILE.TDEPRO"/>
            </contents>
        </node>
        <node type="org.apache.derby.impl.sql.compile.ResultColumnList">
            <contents>
                <member name="indexRow" value="false"/>
            </contents>
            <node type="org.apache.derby.impl.sql.compile.ResultColumn">
                <contents>
                    <member name="exposedName" value="DPCDST"/>
                    <member name="name" value="DPCDST"/>
                    <member name="tableName" value="null"/>
                    <member name="isDefaultColumn" value="false"/>
                    <member name="wasDefaultColumn" value="false"/>
                    <member name="isNameGenerated" value="false"/>
                    <member name="sourceTableName" value="null"/>
                    <member name="type" value="null"/>
                    <member name="columnDescriptor" value="null"/>
                    <member name="isGenerated" value="false"/>
                    <member name="isGeneratedForUnmatchedColumnInInsert" value="false"/>
                    <member name="isGroupingColumn" value="false"/>
                    <member name="isReferenced" value="false"/>
                    <member name="isRedundant" value="false"/>
                    <member name="virtualColumnId" value="1"/>
                    <member name="resultSetNumber" value="-1"/>
                    <member name="dataTypeServices" value="null"/>
                </contents>
            </node>
            <node type="org.apache.derby.impl.sql.compile.ResultColumn">
                <contents>
                    <member name="exposedName" value="DPCDME"/>
                    <member name="name" value="DPCDME"/>
                    <member name="tableName" value="null"/>
                    <member name="isDefaultColumn" value="false"/>
                    <member name="wasDefaultColumn" value="false"/>
                    <member name="isNameGenerated" value="false"/>
                    <member name="sourceTableName" value="null"/>
                    <member name="type" value="null"/>
                    <member name="columnDescriptor" value="null"/>
                    <member name="isGenerated" value="false"/>
                    <member name="isGeneratedForUnmatchedColumnInInsert" value="false"/>
                    <member name="isGroupingColumn" value="false"/>
                    <member name="isReferenced" value="false"/>
                    <member name="isRedundant" value="false"/>
                    <member name="virtualColumnId" value="2"/>
                    <member name="resultSetNumber" value="-1"/>
                    <member name="dataTypeServices" value="null"/>
                </contents>
            </node>
            <node type="org.apache.derby.impl.sql.compile.ResultColumn">
                <contents>
                    <member name="exposedName" value="DPCDPR"/>
                    <member name="name" value="DPCDPR"/>
                    <member name="tableName" value="null"/>
                    <member name="isDefaultColumn" value="false"/>
                    <member name="wasDefaultColumn" value="false"/>
                    <member name="isNameGenerated" value="false"/>
                    <member name="sourceTableName" value="null"/>
                    <member name="type" value="null"/>
                    <member name="columnDescriptor" value="null"/>
                    <member name="isGenerated" value="false"/>
                    <member name="isGeneratedForUnmatchedColumnInInsert" value="false"/>
                    <member name="isGroupingColumn" value="false"/>
                    <member name="isReferenced" value="false"/>
                    <member name="isRedundant" value="false"/>
                    <member name="virtualColumnId" value="3"/>
                    <member name="resultSetNumber" value="-1"/>
                    <member name="dataTypeServices" value="null"/>
                </contents>
            </node>
        </node>
    </node>
</queryTree>

Re: R: R: using ASTParser and TreeWalker

Reply via email to