[jira] Created: (DERBY-704) Large page cache kills initial performance
Large page cache kills initial performance -- Key: DERBY-704 URL: http://issues.apache.org/jira/browse/DERBY-704 Project: Derby Type: Bug Components: Services, Performance Versions: 10.1.2.2, 10.1.3.0, 10.2.0.0, 10.1.2.1, 10.1.2.0, 10.1.1.2, 10.1.1.1, 10.1.1.0 Environment: All platforms Reporter: Knut Anders Hatlen Assigned to: Knut Anders Hatlen Fix For: 10.2.0.0 When the page cache is large the performance gets lower as the page cache is being filled. As soon as the page cache is filled, the throughput increases. In the period with low performance, the CPU usage is high, and when the performance increases the CPU usage is lower. This behaviour is caused by the algorithm for finding free slots in the page cache. If there are invalid pages in the page cache, it will be scanned to find one of those pages. However, when multiple clients access the database, the invalid pages are often already taken. This means that the entire page cache will be scanned, but no free invalid page is found. Since the scan of the page cache is synchronized on the cache manager, all other threads that want to access the page cache have to wait. When the page cache is large, this will kill the performance. When the page cache is full, this is not a problem, as there will be no invalid pages. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-704) Large page cache kills initial performance
[ http://issues.apache.org/jira/browse/DERBY-704?page=all ] Knut Anders Hatlen updated DERBY-704: - Attachment: DERBY-704.diff derbyall_report.txt Attached patch and report from running derbyall (no tests failed). The patch modifies the search for a free slot in the page cache (org.apache.derby.impl.services.cache.Clock.findFreeItem()) in the following way: 1) find out how many invalid pages there are in the page cache 2) check whether pages that are skipped in the search are invalid 3) if the number of skipped invalid pages equals the number of invalid pages, stop the search for invalid pages Could someone please review the patch? Thanks. Large page cache kills initial performance -- Key: DERBY-704 URL: http://issues.apache.org/jira/browse/DERBY-704 Project: Derby Type: Bug Components: Services, Performance Versions: 10.1.2.2, 10.1.3.0, 10.2.0.0, 10.1.2.1, 10.1.2.0, 10.1.1.2, 10.1.1.1, 10.1.1.0 Environment: All platforms Reporter: Knut Anders Hatlen Assignee: Knut Anders Hatlen Fix For: 10.2.0.0 Attachments: DERBY-704.diff, derbyall_report.txt When the page cache is large the performance gets lower as the page cache is being filled. As soon as the page cache is filled, the throughput increases. In the period with low performance, the CPU usage is high, and when the performance increases the CPU usage is lower. This behaviour is caused by the algorithm for finding free slots in the page cache. If there are invalid pages in the page cache, it will be scanned to find one of those pages. However, when multiple clients access the database, the invalid pages are often already taken. This means that the entire page cache will be scanned, but no free invalid page is found. Since the scan of the page cache is synchronized on the cache manager, all other threads that want to access the page cache have to wait. When the page cache is large, this will kill the performance. When the page cache is full, this is not a problem, as there will be no invalid pages. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: RowLocation lifetime
Suresh Thalamati wrote: Andreas Korneliussen wrote: Mike Matrigali wrote: snip I would think there are multiple ways of adressing this issue: 1 We could make the store graciously handle the situation if the RowLocation points to a deleted+purged row, by returning false if the RowLocation is invalid, (and from the caller we can give an exception) This may not be good option, because purged row location can potentially be used by an another insert from a different transaction. Maybe I misunderstood, however I assumed the RowLocation would not be reused as long as there is a table intent lock on the table. Therefore the insert from a different transaction would need to use another RowLocation. Mike Matrigali wrote: 2) It can be purged (requires at least TABLE IX, ROW X locking) o all physical evidence of the row is removed from table, both internal and external operations on this row will fail. Only committed deleted rows are purged. Note this will never happen if you have some sort of lock on the row as the requested X lock is always requested in a system only transaction. o the actual RowLocation will not be reused while at least some sort of table level intent lock is held. --Andreas
[jira] Updated: (DERBY-704) Large page cache kills initial performance
[ http://issues.apache.org/jira/browse/DERBY-704?page=all ] Knut Anders Hatlen updated DERBY-704: - Attachment: throughput.png cpu-usage.png Attached graphs showing the throughput and CPU usage running with and without the patch. The graphs show the average of seven runs, throughput and CPU usage were reported every 30 seconds. 12 clients were running an update-intensive load on the database. The database had 10 GB of user data, and the page cache size was 512 MB. The CPU usage is relatively low because the tests were run on an 8-CPU machine. The first 30 minutes the CPU usage was higher and the throughput significantly lower without the patch than with the patch. Large page cache kills initial performance -- Key: DERBY-704 URL: http://issues.apache.org/jira/browse/DERBY-704 Project: Derby Type: Bug Components: Services, Performance Versions: 10.1.2.2, 10.1.3.0, 10.2.0.0, 10.1.2.1, 10.1.2.0, 10.1.1.2, 10.1.1.1, 10.1.1.0 Environment: All platforms Reporter: Knut Anders Hatlen Assignee: Knut Anders Hatlen Fix For: 10.2.0.0 Attachments: DERBY-704.diff, cpu-usage.png, derbyall_report.txt, throughput.png When the page cache is large the performance gets lower as the page cache is being filled. As soon as the page cache is filled, the throughput increases. In the period with low performance, the CPU usage is high, and when the performance increases the CPU usage is lower. This behaviour is caused by the algorithm for finding free slots in the page cache. If there are invalid pages in the page cache, it will be scanned to find one of those pages. However, when multiple clients access the database, the invalid pages are often already taken. This means that the entire page cache will be scanned, but no free invalid page is found. Since the scan of the page cache is synchronized on the cache manager, all other threads that want to access the page cache have to wait. When the page cache is large, this will kill the performance. When the page cache is full, this is not a problem, as there will be no invalid pages. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: RowLocation lifetime
Mike Matrigali wrote: i get confused when speaking about isolation level, update/read only result sets, and underlying sql query of the result set. I don't know if one scrollable result sets are dependent on some sort of isolation level. With respect to straight embedded server execution of SQL, it is fine to run with read-uncommitted level - but any actual update done on a row will get an X lock held to end transaction. At least from this level an SQL operation is never failed dependent on the isolation level. I don't remember if U locks are requested in read uncommitted mode, but definitely X locks are requested when the actual update is done. Note that all discussions of locking should specify under which isolation level the system is running. I assumed read commited for the below discussion as it is the default. The discussion was intended for read-committed and read-uncommitted, since for other isolation levels, the rows for which we use the RowLocation would be locked, and cannot be deleted or purged by another transaction. Also, if we delete the row in our own transaction, it will not be purged, since it is locked with an exclusive lock. I think you are allowed to do updates in read-uncommitted, however when you read, you do not set read-locks, so you can read uncommitted data. Andreas Daniel John Debrunner wrote: Andreas Korneliussen wrote: 2 Or we could make all scrollable updatable resultsets set read-locks or updatelocks on every row, for all isolation levels (including read-uncommitted) I think updates are not allowed in read-uncommitted mode, so we should not be getting locks in read-uncommitted. Dan.
[jira] Commented: (DERBY-704) Large page cache kills initial performance
[ http://issues.apache.org/jira/browse/DERBY-704?page=comments#action_12357574 ] Knut Anders Hatlen commented on DERBY-704: -- Sorry, I forgot the output from svn stat. M java/engine/org/apache/derby/impl/services/cache/Clock.java Large page cache kills initial performance -- Key: DERBY-704 URL: http://issues.apache.org/jira/browse/DERBY-704 Project: Derby Type: Bug Components: Services, Performance Versions: 10.1.1.0, 10.2.0.0, 10.1.2.0, 10.1.1.1, 10.1.1.2, 10.1.2.1, 10.1.3.0, 10.1.2.2 Environment: All platforms Reporter: Knut Anders Hatlen Assignee: Knut Anders Hatlen Fix For: 10.2.0.0 Attachments: DERBY-704.diff, cpu-usage.png, derbyall_report.txt, throughput.png When the page cache is large the performance gets lower as the page cache is being filled. As soon as the page cache is filled, the throughput increases. In the period with low performance, the CPU usage is high, and when the performance increases the CPU usage is lower. This behaviour is caused by the algorithm for finding free slots in the page cache. If there are invalid pages in the page cache, it will be scanned to find one of those pages. However, when multiple clients access the database, the invalid pages are often already taken. This means that the entire page cache will be scanned, but no free invalid page is found. Since the scan of the page cache is synchronized on the cache manager, all other threads that want to access the page cache have to wait. When the page cache is large, this will kill the performance. When the page cache is full, this is not a problem, as there will be no invalid pages. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: RowLocation lifetime
Mike Matrigali wrote: null pointer is a bug, please report as a separate JIRA,not sure what is going on. Note that while it is convenient for testing purposes to use the SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE to cause the purging, purging of rows can happen at any time after a delete has been committed. The timing depends on a number of factors and the timing may change in the future - so one should assume that as soon as a row is deleted and committed it may be purged. Andreas Korneliussen wrote: Mike Matrigali wrote: From the store point of view, 3 things can happen to RowLocations in heap tables: 1) It can be deleted (requires at least TABLE IX, ROW X locking) o internal to store it is just marked deleted o external to store requests for any operation on this row will fail. Note that your cursor can experience this no matter what locking you do, as it is always possible for another statement in your transaction to do the delete. 2) It can be purged (requires at least TABLE IX, ROW X locking) o all physical evidence of the row is removed from table, both internal and external operations on this row will fail. Only committed deleted rows are purged. Note this will never happen if you have some sort of lock on the row as the requested X lock is always requested in a system only transaction. o the actual RowLocation will not be reused while at least some sort of table level intent lock is held. 3) It can be reused (requires a table level X lock) o basically as part of a compress all rows can be shuffled in any way. A former RowLocation can now point to a completely different row. So as you point out, your implementation can have serious problems with cursors held over commit. This is why in current usage of cursors over commit the only safe thing to do is to ask for the next row location and use it. Please make sure to consider the delete/purge cases also. One case that often causes problems is a transaction deleting a row that is locked by it's own cursor from another statement in the same connection. Yes, we need to consider those cases. It seems that the store is capable of graciously handle that the row get deleted (i.e by its own transaction). If the transaction later tries to update the deleted row using the resultset, the store call will return false indicating that the row was not updated. The deleted row will not be purged as long as the transaction is open. However in read-committed/read-uncommitted mode, a row read by the cursor, can be deleted by another transaction, and then purged. It seems that the store does not handle an update of a deleted+purged record. On our prototype impl., I get a get a NullPointerException from the store in this case. It comes in GenericConglomerateController.replace(..)). I would think there are multiple ways of adressing this issue: 1 We could make the store graciously handle the situation if the RowLocation points to a deleted+purged row, by returning false if the RowLocation is invalid, (and from the caller we can give an exception) It seems like the ConglomerateController.replace() function should throw an exception (other than null pointer) if it is called with a non-existent RowLocation, but I could be convinced returning false is ok. The problem I have is that store really has no way to tell the difference between a BAD RowLocation input and one which was purged. Yes, maybe the store should throw an exception if the RowLocation is invalid. Seen from the user perspective, I think the update should fail the same way whether the row has been deleted+purged or only deleted Currently the function returns false if the row has been deleted. I ran a check on all places in the code tree that the ConglomerateCongtroller.replace(..) method is called. I found that the return value is silently ignored all places, except for in the store unit-tests. This means that an updateRow() on a deleted record, would simply return silently (using the optimistic concurrency approach). After the transaction has committed, the row would remain deleted. I will file a JIRA for the NullPointerException case. --Andreas
Re: New features for next release .... (Was: Grant and Revoke ... DERBY-464...)
Hi, Daniel == Daniel John Debrunner [EMAIL PROTECTED] wrote: Daniel This would be great to be on a wiki page, including links to the Jira Daniel entries with the functional specs. I'm losing track of which functional Daniel specs are out there, just saw in some reply that Dag has posted a Daniel functional spec on RowId (I think), must have missed the original note. The func spec I posted was for scrollable, updatable result sets, attached to JIRA 690 as http://issues.apache.org/jira/secure/attachment/12320548/sur-proposal.txt Dag
[jira] Commented: (DERBY-587) Providing JDBC 4.0 support for derby
[ http://issues.apache.org/jira/browse/DERBY-587?page=comments#action_12357595 ] Rick Hillegas commented on DERBY-587: - I have successfully built this patch under jdk1.4 and jdk1.6. For me derbyall runs cleanly under jdk1.4. Under jdk1.6 the new jdbc4 tests run cleanly. Satheesh, when you have a moment, could you verify that this submission satisifies the copyright and formatting issues you raised? If so, and there are no new issues, then I think this is ready to be committed. Providing JDBC 4.0 support for derby Key: DERBY-587 URL: http://issues.apache.org/jira/browse/DERBY-587 Project: Derby Type: New Feature Components: JDBC Versions: 10.2.0.0 Reporter: V.Narayanan Assignee: V.Narayanan Priority: Minor Fix For: 10.2.0.0 Attachments: jdbc4.0.sxw, jdbc4.diff -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
Daniel John Debrunner wrote: ... How much demand for this type, as you've described is there? Are many Java applications using byte for fields? My guess would have been that any requests for TINYINT would have been due to existing database applications. I think Lance is right: we can see demand for this type in the fact that a relatively new database like MySQL bothered to add it. One potential issue is that we are creating a non-standard datatype that is in conflict with the existing type of the same name by SQL Server and Sybase. Will this cause more problems for users of Derby as they try to migrate off those databases? Of course in some ways it's not much better if we go 0-255, as then it's in conflict with MySQL. Though I would be interested to know if the new strict flag in MySQL has any effect on TINYINT. Right, we need both signed and unsigned bytes to ease all of these migration paths. That's the appeal of starting out with a signed TINYINT and adding the UNSIGNED keyword later on. Cheers, -Rick
[jira] Commented: (DERBY-506) Implement Statement.setQueryTimeout in the Client Driver
[ http://issues.apache.org/jira/browse/DERBY-506?page=comments#action_12357597 ] Oyvind Bakksjo commented on DERBY-506: -- Sendingjava/client/org/apache/derby/client/am/PreparedStatement.java Sendingjava/client/org/apache/derby/client/am/Statement.java Sendingjava/drda/org/apache/derby/impl/drda/DRDAConnThread.java Sending java/testing/org/apache/derbyTesting/functionTests/suites/DerbyNetClient.exclude Sending java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/SetQueryTimeoutTest.java Transmitting file data . Committed revision 344147. Implement Statement.setQueryTimeout in the Client Driver Key: DERBY-506 URL: http://issues.apache.org/jira/browse/DERBY-506 Project: Derby Type: New Feature Components: JDBC Versions: 10.1.1.0 Reporter: Oyvind Bakksjo Assignee: Oyvind Bakksjo Attachments: DERBY-506.diff, DERBY-506.status Currently, the Embedded Driver supports Statement.setQueryTimeout(), but the Client Driver does not. The Client Driver should be enhanced and match the Embedded Driver. For this, we need to transfer the timeout value from the client to the server, preferably without a separate round-trip. I have some loose thoughts on how to do this: * If the client has set a timeout value for a statement, prepend the (DRDA) EXCSQLSTT command with an EXCSQLSET command which contains the timeout value; conceptually a SET STATEMENT TIMEOUT seconds (this does not mean that we need to extend the Derby grammar; only the Network Server needs to understand this DRDA EXCSQLSET command). * In DRDAConnThread.parseEXCSQLSETobjects() on the server side, recognize the SET STATEMENT TIMEOUT text, parse the timeout value and remember it for the coming EXCSQLSTT command. Do NOT invoke executeUpdate() with the SET statement [see note below]. * In DRDAConnThread.parseEXCSQLSTT(), check if a timeout value has been set; if so, use it (by setting the timeout value on the server-side Statement object before calling execute/executeQuery). -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (DERBY-506) Implement Statement.setQueryTimeout in the Client Driver
[ http://issues.apache.org/jira/browse/DERBY-506?page=all ] Oyvind Bakksjo resolved DERBY-506: -- Resolution: Fixed Had to disable some testing because of DERBY-694 Implement Statement.setQueryTimeout in the Client Driver Key: DERBY-506 URL: http://issues.apache.org/jira/browse/DERBY-506 Project: Derby Type: New Feature Components: JDBC Versions: 10.1.1.0 Reporter: Oyvind Bakksjo Assignee: Oyvind Bakksjo Attachments: DERBY-506.diff, DERBY-506.status Currently, the Embedded Driver supports Statement.setQueryTimeout(), but the Client Driver does not. The Client Driver should be enhanced and match the Embedded Driver. For this, we need to transfer the timeout value from the client to the server, preferably without a separate round-trip. I have some loose thoughts on how to do this: * If the client has set a timeout value for a statement, prepend the (DRDA) EXCSQLSTT command with an EXCSQLSET command which contains the timeout value; conceptually a SET STATEMENT TIMEOUT seconds (this does not mean that we need to extend the Derby grammar; only the Network Server needs to understand this DRDA EXCSQLSET command). * In DRDAConnThread.parseEXCSQLSETobjects() on the server side, recognize the SET STATEMENT TIMEOUT text, parse the timeout value and remember it for the coming EXCSQLSTT command. Do NOT invoke executeUpdate() with the SET statement [see note below]. * In DRDAConnThread.parseEXCSQLSTT(), check if a timeout value has been set; if so, use it (by setting the timeout value on the server-side Statement object before calling execute/executeQuery). -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (DERBY-506) Implement Statement.setQueryTimeout in the Client Driver
[ http://issues.apache.org/jira/browse/DERBY-506?page=all ] Oyvind Bakksjo closed DERBY-506: Implement Statement.setQueryTimeout in the Client Driver Key: DERBY-506 URL: http://issues.apache.org/jira/browse/DERBY-506 Project: Derby Type: New Feature Components: JDBC Versions: 10.1.1.0 Reporter: Oyvind Bakksjo Assignee: Oyvind Bakksjo Attachments: DERBY-506.diff, DERBY-506.status Currently, the Embedded Driver supports Statement.setQueryTimeout(), but the Client Driver does not. The Client Driver should be enhanced and match the Embedded Driver. For this, we need to transfer the timeout value from the client to the server, preferably without a separate round-trip. I have some loose thoughts on how to do this: * If the client has set a timeout value for a statement, prepend the (DRDA) EXCSQLSTT command with an EXCSQLSET command which contains the timeout value; conceptually a SET STATEMENT TIMEOUT seconds (this does not mean that we need to extend the Derby grammar; only the Network Server needs to understand this DRDA EXCSQLSET command). * In DRDAConnThread.parseEXCSQLSETobjects() on the server side, recognize the SET STATEMENT TIMEOUT text, parse the timeout value and remember it for the coming EXCSQLSTT command. Do NOT invoke executeUpdate() with the SET statement [see note below]. * In DRDAConnThread.parseEXCSQLSTT(), check if a timeout value has been set; if so, use it (by setting the timeout value on the server-side Statement object before calling execute/executeQuery). -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (DERBY-705) Re-enable test which was disabled in DERBY-506 commit
Re-enable test which was disabled in DERBY-506 commit - Key: DERBY-705 URL: http://issues.apache.org/jira/browse/DERBY-705 Project: Derby Type: Test Components: Test Reporter: Oyvind Bakksjo Priority: Trivial Testing of concurrent executions on same connection was disabled in DERBY-506 commit because of DERBY-694 bug. Re-enable when DERBY-694 is fixed. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: RowLocation lifetime
Hi Mike, It appears that RowLocations are stable under some combinations of isolation levels and lock granularities but that, in general, a RowLocation isn't guaranteed to be stable even for the duration of a statement. Is there some other, more stable handle on a heap row? By more stable I mean something which uniquely identifies the row across statements, transactions, or connections? Thanks, -Rick
[jira] Commented: (DERBY-47) Some possible improvements to IN optimization
[
http://issues.apache.org/jira/browse/DERBY-47?page=comments#action_12357601 ]
Kevin Hore commented on DERBY-47:
-
I have also found this issue to be a problem and would like to know whether
there are any plans to fix it?
What follows is a copy of discussion from the derby-users mailing list form
2005/11/11 Poor query optimizer choices is making Derby unusable for large
tables, it describes how the same behaviour is causing problems for a query
with a GROUP BY clause:
---
Hi Kevin,
Kevin Hore wrote:
i) Does anyone have any plans to fix this problem?
Can you file an enhancement request for this? I think Derby could
improve it's handling of OR/IN clauses. Many databases don't optimize OR
clauses as much as possible, though some do better than others. It would
be great if Derby could internally process this as two different scans
(one for 'CONTACT' and another for 'ADD') and then combine the results.
Some databases can do this. However, the workaround suggested by Jeff L.
does this, though you have to rewrite the query.
Satheesh
ii) In the meantime, are there any work-arounds? I'd appreciate any
suggestions that would decrease the execution time of my second query
below (the one with with two search terms). Likewise, any general
strategies for avoiding this problem with IN clauses would be
appreciated.
PROBLEM DESCRIPTION
Consider the table:
CREATE TABLE tblSearchDictionary
(
ObjectId int NOT NULL,
ObjectType int NOT NULL,
Word VARCHAR(64) NOT NULL,
WordLocation int NOT NULL,
CONSTRAINT CONSd0e222 UNIQUE (ObjectId,ObjectType,Word,WordLocation)
);
This table has an index on each of the four columns, it also has the
unique index across all four columns as defined above:
CREATE INDEX tblSearchDictionaryObjectId ON tblSearchDictionary
(ObjectId);
CREATE INDEX tblSearchDictionaryObjectType ON tblSearchDictionary
(ObjectType);
CREATE INDEX tblSearchDictionaryWord ON tblSearchDictionary (Word);
CREATE INDEX tblSearchDictionaryWordLocation ON tblSearchDictionary
(WordLocation);
The table contains about 260,000 rows.
The following query selects all rows that match instances of string in
the Word column. It sums the WordLocation column having grouped by the
ObjectId column.
SELECT ObjectId, SUM(WordLocation) AS Score
FROM tblSearchDictionary
WHERE Word = 'CONTACT'
GROUP BY ObjectId;
On my machine this will usually complete in an acceptable time of
around 200ms.
Now consider the following query which adds a second search term on
the same column.
SELECT ObjectId, SUM(WordLocation) AS Score
FROM tblSearchDictionary
WHERE Word = 'CONTACT' OR Word = 'ADD'
GROUP BY ObjectId;
This second query usually takes around 1ms on my machine. My
understanding from the Derby optimizer docs and DERBY-47 is that this
is because Derby is re-writing the query along the following lines,
and then choosing to do a table scan:
SELECT ObjectId, SUM(WordLocation) AS Score
FROM tblSearchDictionary
WHERE
Word IN ('CONTACT', 'ADD')
AND Word = 'ADD'
AND Word = 'CONTACT'
GROUP BY ObjectId;
The plan for the first query indicates that the tblSearchDictionaryWord
index is used to perform an index scan. However, the plan for the second
query indicates that the majority of the additional time is taken
performing a table scan over the entire table, instead of making use of
the indexes available. Our application uses IN quite frequently, so
this optimizer behaviour would seem to present a significant problem.
---QUERY PLAN FOR FIRST QUERY
Statement Name:
null
Statement Text:
SELECT
ObjectId,
SUM(WordLocation) AS Score
FROM tblSearchDictionary
WHERE
Word = 'CONTACT'
GROUP BY
ObjectId
Parse Time: 0
Bind Time: 0
Optimize Time: 16
Generate Time: 0
Compile Time: 16
Execute Time: 0
Begin Compilation Timestamp : 2005-11-11 12:28:52.765
End Compilation Timestamp : 2005-11-11 12:28:52.781
Begin Execution Timestamp : 2005-11-11 13:08:15.406
End Execution Timestamp : 2005-11-11 13:08:15.406
Statement Execution Plan Text:
Project-Restrict ResultSet (5):
Number of opens = 1
Rows seen = 93
Rows filtered = 0
restriction = false
projection = true
constructor time (milliseconds) = 0
open time (milliseconds) = 0
next time (milliseconds) = 0
close time (milliseconds) = 0
restriction time (milliseconds) = 0
projection time (milliseconds) = 0
optimizer estimated row count:1.00
optimizer estimated cost: 226.00
Source result set:
Grouped Aggregate ResultSet:
Number of opens = 1
Rows input = 113
Has distinct aggregate = false
In sorted order = false
Sort information:
Number of rows input=113
Number of rows output=93
Sort type=internal
constructor time (milliseconds) = 0
Re: RowLocation lifetime
Not really. I don't think even a unique key constraint meets the requirements, as the row can be deleted and a different row can be created. Rick Hillegas wrote: Hi Mike, It appears that RowLocations are stable under some combinations of isolation levels and lock granularities but that, in general, a RowLocation isn't guaranteed to be stable even for the duration of a statement. Is there some other, more stable handle on a heap row? By more stable I mean something which uniquely identifies the row across statements, transactions, or connections? Thanks, -Rick
Building jars from 10.1.2.1 source fails: Manifest is invalid
Hi all,
has anyone tried to build the Derby jars based on the 10.1.2.1 (10.1.2
Release Candidate #2) source code archive? (We are supposed to be able
to do this, right?)
I downloaded db-derby-10.1.2.1-src.tar.gz from
http://people.apache.org/~kmarsden/derby10.1.2.1.330608/ and extracted
the archive.
Running ant all goes just fine - however, ant buildjars fails,
saying that the jars/sane/lists.smf.mf manifest is invalid. Ant
complains that 'Manifest sections should start with a Name attribute
and not Sealed' (see attachment ant_buildjars_error.txt).
I compared the smf.mf of the 10.1.2.1 build with the same file in my
subversion repository, and the only difference was that the 10.1.2.1
manifest has an extra blank line between Bundle-Version and the first
Sealed: true attribute in the manifest, which apparently causes the
failure (see attachment smf.mf).
The lonely Sealed: true attribute is obviously supposed to be in the
main section of the manifest.
http://java.sun.com/j2se/1.4.2/docs/guide/jar/jar.html states that
No attribute in this section can have its name equal to Name. This
section is terminated by an empty line. Thus, the extra empty line is
causing the failure.
A google search led me to a thread in the mailing list archive of Apache
Gump: http://tinyurl.com/742ln.
It seems that Andrew has come across the same build error fairly
recently, and that this is/was a subversion problem.
So, Andrew, any ideas why this error appears when I try to build the
jars based on the code from the 10.1.2.1 source archive?
My build environment:
Solaris 10 (x86)
ANT_HOME=/usr/local/share/java/apache-ant-1.6.2
PATH=${ANT_HOME}/bin:$PATH
JAVA_HOME=/usr/local/java/jdk1.4
ant.properties:
j14lib=/usr/local/java/jdk1.4/jre/lib
j13lib=/usr/local/java/jdk1.3/jre/lib
proceed=false
sane=true
--
John
derbyjarwithoutosgi:
[jar] Building jar:
/home/je159969/apache/Derby/snapshots/10.1.2.1_RC_SVN330608_ownBuild/jars/sane/derby.jar
[jar] Manifest is invalid: Manifest sections should start with a Name
attribute and not Sealed
BUILD FAILED
/home/je159969/apache/Derby/snapshots/10.1.2.1_RC_SVN330608_ownBuild/build.xml:775:
The following error occurred while executing this line:
/home/je159969/apache/Derby/snapshots/10.1.2.1_RC_SVN330608_ownBuild/build.xml:854:
Invalid Manifest:
/home/je159969/apache/Derby/snapshots/10.1.2.1_RC_SVN330608_ownBuild/jars/sane/lists/smf.mf
Total time: 7 seconds
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.6.2
Created-By: 1.4.2_02-b03 (Sun Microsystems Inc.)
Bundle-Vendor: Apache Software Foundation
Bundle-Name: Apache Derby 10.1
Bundle-Version: 10.1.201.exported
Sealed: true
Name: org/apache/derby/impl/tools/sysinfo/
Sealed: false
Name: org/apache/derby/iapi/services/context/
Sealed: false
Name: org/apache/derby/iapi/services/info/
Sealed: false
Name: org/apache/derby/jdbc/
Sealed: false
Name: org/apache/derby/info/
Sealed: false
Name: org/apache/derby/iapi/error/
Sealed: false
Name: org/apache/derby/iapi/services/i18n/
Sealed: false
Name: org/apache/derby/impl/jdbc/
Sealed: false
Name: org/apache/derby/iapi/tools/i18n/
Sealed: false
Name: org/apache/derby/loc/
Sealed: false
Name: org/apache/derby/tools/
Sealed: false
Re: New features for next release .... (Was: Grant and Revoke ... DERBY-464...)
I like Lance's suggestion and would like to propose it as a general policy. I think that this will handle Army's XML work as well as the other new 10.2 datatypes: If you add a new datatype to a server release (e.g., BOOLEAN or XML), then you must specify the following: 1) A legacy datatype to which the new type maps when the server talks to old clients. The legacy type is some datatype in JDBC 2.0 java.sql.Types. This is the type which old clients see in DatabaseMetaData and ResultSetMetaData. 2) A pair of ResultSet.get() and PreparedStatement.set() methods which old clients can use to access the new datavalues. These must be get() and set() methods which appear in JDBC 2.0 ResultSet and PreparedStatement. They should be the get() and set() methods most natural to the legacy datatype in (1). These methods determine how the datavalues flow across DRDA. I'm attaching a table which shows mappings for XML and the other new 10.2 datatypes. I have omitted ROWID right now because I don't understand it yet. Cheers, -Rick Lance J. Andersen wrote: Satheesh Bandaram wrote: Hi Rick, Rick Hillegas wrote: We will probably have to collaborate here. If you select an XML column, the ResultSetMetaData needs to say that the column is of type java.sql.SQLXML. This, at least, was the consensus of the community which prevented me from re-enabling the BOOLEAN datatype a couple months ago: Kathey and David pointed out that it was not OK for ResultSetMetaData to report a boolean column's type as SMALLINT. Similarly, it's not going to be OK for ResultSetMetaData to report the type of an XML column as java.sql.LONGVARCHAR. It doesn't seem right to expose a JDBC 4.0 type to a JDBC 3.0 client. What good would that do, since JDBC 3.0 clients are probably written to expect JDBC 3.0 functionality only? I wonder if we might even break GUIs and other generic tools that expect 3.0 types only. Since we may be close to enabling full JDBC 4.0 in the near term, why not expose only JDBC 3.0 types to 3.0 clients? That is to expose XML type as a CLOB under JDBC 3.0 and as SQLXML for a JDBC 4.0 client. This would expose a new type to Derby server, XML, to JDBC by mapping it to it's closest match. Remember the TINYINT discussion and Dan's point? A JDBC 3 driver should not return SQLXML as a type in ResultSetMetaData. I am not sure TINYINT is the same discussion though as JDBC has provided support for this since JDBC 1.0.x. A JDBC 3 driver should only return results defined in classes for the given version of JDBC and since JDBC 3 did not have SQLXML in java.sql.Types, you need to return a different value. I don't remember what was discussed about this during the BOOLEAN discussion... but if JDBC already supports a boolean type, that would seem correct to map it to, rather than say SMALLINT. Correct, if the datatype is in Types.java and the backend datatype is also there, they should map to the same (unless there is something major league wrong with what the meaning of the type is on the backend) Regards Lance Satheesh Title: New 10.2 Derby Datatypes New 10.2 Derby Datatypes This table describes how to map new 10.2 datatypes to legacy types supported by old Derby clients under JDBC 2.0: New type Legacy type Legacy get() Legacy set() TINYINT SMALLINT ResultSet.getShort() PreparedStatement.setShort() BOOLEAN SMALLINT ResultSet.getShort() PreparedStatement.setShort() NCHAR CHAR ResultSet.getString() PreparedStatement.setString() NVARCHAR VARCHAR ResultSet.getString() PreparedStatement.setString() LONGNVARCHAR LONGVARCHAR ResultSet.getCharacterStream() PreparedStatement.setCharacterStream() NCLOB CLOB ResultSet.getCharacterStream() PreparedStatement.setCharacterStream() SQLXML CLOB ResultSet.getCharacterStream() PreparedStatement.setCharacterStream()
[jira] Commented: (DERBY-614) Execution failed because of a Distributed Protocol Error
[
http://issues.apache.org/jira/browse/DERBY-614?page=comments#action_12357583 ]
Kathey Marsden commented on DERBY-614:
--
I wanted to check in on this issue. I wanted to make sure that my comment that
it would be good to get input wasn't stalling your effort. I think the approach
you proposed is a good one.
Execution failed because of a Distributed Protocol Error
Key: DERBY-614
URL: http://issues.apache.org/jira/browse/DERBY-614
Project: Derby
Type: Bug
Components: Network Server
Versions: 10.1.1.0
Environment: Linux 2.4, Sun JDK 1.4.2_07, full SYSINFO will be attached.
Reporter: Bryan Pendleton
Attachments: clientSideTrace.txt, clientStack.txt, derby-614.zip,
derbyTrace.txt, query.txt, serverSideTrace.txt, sysinfo.derby
I am intermittently receiving Distributed Protocol Error exceptions in my
client code. Simultaneously,
my derby.log is recording Distributed Protocol Error exceptions on the server
side.
I cannot reliably reproduce this problem.
However, the problem always occurs in the same section of my application, so
I can at least
describe that section of my application. (I'll do so in an attachment).
Here is the error I receive on the server side. Sometimes, the CODPNT is 2116
and the
Error Code Value is 1d, rather than 2114 and e.
2005-10-05 02:10:23.663 GMT Thread[DRDAConnThread_2,5,main] (DATABASE =
BuildFar
m), (DRDAID = GA0A0026.P7E6-4182154075488704215{136532}), Execution failed
becau
se of a Distributed Protocol Error: DRDA_Proto_SYNTAXRM; CODPNT arg = 2114;
Er
ror Code Value = e
Execution failed because of a Distributed Protocol Error:
DRDA_Proto_SYNTAXRM;
CODPNT arg = 2114; Error Code Value = e
2005-10-05 02:10:23.663 GMT Thread[DRDAConnThread_2,5,main] (DATABASE =
BuildFar
m), (DRDAID = GA0A0026.P7E6-4182154075488704215{136532}), null
null
org.apache.derby.impl.drda.DRDAProtocolException
at org.apache.derby.impl.drda.DRDAConnThread.throwSyntaxrm(Unknown
Sourc
e)
at org.apache.derby.impl.drda.DRDAConnThread.missingCodePoint(Unknown
So
urce)
at org.apache.derby.impl.drda.DRDAConnThread.parseCNTQRY(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.splitQRYDTA(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.writeFDODTA(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.writeQRYDTA(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Sou
rce)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: q re dataSourcePermissions_net and DERBY-410
Myrna van Lunteren wrote:
Hi,
I'm working on DERBY-413, trying to remove the hardcoded 'localhost' from
the tests, and found this comment and subsequent line in the test
derbynet/dataSourcePermissions_net.java:
--
if (TestUtil.isJCCFramework())
{
attrs.setProperty(driverType,4);
/**
* As per the fix of derby-410
* servername should now default to localhost
*/
attrs.setProperty(serverName,localhost);
}
---
Now, if the serverName defaults to localhost, why are we passing it in?
?
For JCC it has to be set, so it is set if
(TestUtil.isJCCFramework()). The comment might be clearer if it it said
servername should now default to localhost for derbyclient but still needs to
be set for JCC.
Re: [jira] Commented: (DERBY-699) Allow schema and savepoint names starting with SYS
Hi! Satheesh Bandaram (JIRA) wrote (2005-11-10 19:22:03): Satheesh Bandaram commented on DERBY-699: - Derby needs to treat many schemas starting with SYS.. as system schemas... like SYSCAT, SYS, SYSFUN, SYSSTAT, SYSPROC, SYSIBM, SYSCS_DIAG, SYSCS_UTIL. It may be possible to add future system schemas too, like SYSFUN was added recently. Any change to allow user SYS schemas needs to be made very carefully. I agree that Derby needs to reserve a part of the namespace for future system schema names. But I find it unreasonable to reserve all names starting with SYS because SYSTEM is a very common word and that this would disallow schema names like SYSTEM_DIAGNOSTICS, SYSTEM_STATUS etc. A common prefix like this might be a hurdle for porting existing schemas and applications to Derby (that's the way I discovered the restriction). I also see from the comments in the code that this restriction was added as a DB2-compliance, and think that we should revert this restriction. The best approach would probably be to add a non-standard character to the identifiers so that a non-standard prefix could be used. E.g. SYS$. -- Bernt Marius Johnsen, Database Technology Group, Sun Microsystems, Trondheim, Norway pgpYXHPUCsVFj.pgp Description: PGP signature
Re: Building jars from 10.1.2.1 source fails: Manifest is invalid
On Nov 14, 2005, at 9:00 AM, John Embretsen wrote:
has anyone tried to build the Derby jars based on the 10.1.2.1 (10.1.2
Release Candidate #2) source code archive? (We are supposed to be able
to do this, right?)
A google search led me to a thread in the mailing list archive of
Apache
Gump: http://tinyurl.com/742ln.
It seems that Andrew has come across the same build error fairly
recently, and that this is/was a subversion problem.
So, Andrew, any ideas why this error appears when I try to build the
jars based on the code from the 10.1.2.1 source archive?
Yes, there is a problem with svnversion in subversion 1.1.x and
earlier that causes an additional carriage return to be appended to
directories that are reported as 'exported', even when run with the -
n option. The property ${changenumber} which contains the version
reported by subversion then contains an additional carriage return,
which in turn breaks the manifest file format by setting the top-
level Sealed attribute apart from the rest of the top-level
attributes as you noted.
I checked in a fix to the trunk so that Gump could continue to run.
It's a rather simple patch, which you could apply to the build.xml
from the source distribution, attached to the end of this mail.
This problem has been fixed in Subversion 1.2.0 and above. Upgrading
to the latest subversion will also remedy the problem. For this
reason, I would consider it a non-showstopper for holding up the
10.1.2.1 release, but I will merge the fix over to the 10.1 branch
for future releases.
andrew
Index: build.xml
===
--- build.xml (revision 330289)
+++ build.xml (revision 330290)
@@ -700,9 +700,13 @@
property name=derby.jar.topdir value=${basedir}/jars/
mkdir dir=${derby.jar.dir}/
mkdir dir=${derby.jar.dir}/lists/
- loadfile srcFile=${basedir}/changenumber.properties
+loadfile srcFile=${basedir}/changenumber.properties
failonerror=false
- property=changenumber/
+ property=changenumber
+ filterchain
+striplinebreaks/
+ /filterchain
+/loadfile
condition property=changenumber value=???
not
isset property=changenumber/
Re: New features for next release .... (Was: Grant and Revoke ... DERBY-464...)
Rick Hillegas wrote: I like Lance's suggestion and would like to propose it as a general policy. I think that this will handle Army's XML work as well as the other new 10.2 datatypes: If you add a new datatype to a server release (e.g., BOOLEAN or XML), then you must specify the following: Maybe replace 'you must' with 'you can'. If someone has the itch to add a type for embedded only, then I don't think they should be forced to make it work with old or new clients. Also for something like XML datatype, is there any requirement to make it be available (in any form) with old clients? Is it not sufficient to say if you want to use the XML data type you must use the 10.2 client. If someone wants to do that work, that's fine, but I don't see it as a requiement. 1) A legacy datatype to which the new type maps when the server talks to old clients. The legacy type is some datatype in JDBC 2.0 java.sql.Types. This is the type which old clients see in DatabaseMetaData and ResultSetMetaData. Can old clients that are running as JDBC 3.0 see types from JDBC 3.0? Is this just for the network server, how about embedded, e.g. running Derby on JDK 1.3/1.4? 2) A pair of ResultSet.get() and PreparedStatement.set() methods which old clients can use to access the new datavalues. These must be get() and set() methods which appear in JDBC 2.0 ResultSet and PreparedStatement. They should be the get() and set() methods most natural to the legacy datatype in (1). These methods determine how the datavalues flow across DRDA. Just curious as to why specifying the getXXX and setXXX method is required, doesn't it follow from the legacy JDBC type specified? Or is there some deeper thought here I am missing? For example, in your example, with NCLOB can the client use setClob, setString etc? Dan.
[jira] Commented: (DERBY-506) Implement Statement.setQueryTimeout in the Client Driver
[ http://issues.apache.org/jira/browse/DERBY-506?page=comments#action_12357610 ] Satheesh Bandaram commented on DERBY-506: - Thanks for the useful and good patch!! I did a quick review of the code and have couple of comments: 1) Is there any need to keep the am/QueryTimerTask.java still? Since there doesn't seem to be any users of this, may be this class could be dropped? 2) What would happen if a 10.2 client tries to send a SET STATEMENT_TIMEOUT statement to a 10.1 server? There doesn't seem to be any mechanism to prevent unpleasant events. Implement Statement.setQueryTimeout in the Client Driver Key: DERBY-506 URL: http://issues.apache.org/jira/browse/DERBY-506 Project: Derby Type: New Feature Components: JDBC Versions: 10.1.1.0 Reporter: Oyvind Bakksjo Assignee: Oyvind Bakksjo Attachments: DERBY-506.diff, DERBY-506.status Currently, the Embedded Driver supports Statement.setQueryTimeout(), but the Client Driver does not. The Client Driver should be enhanced and match the Embedded Driver. For this, we need to transfer the timeout value from the client to the server, preferably without a separate round-trip. I have some loose thoughts on how to do this: * If the client has set a timeout value for a statement, prepend the (DRDA) EXCSQLSTT command with an EXCSQLSET command which contains the timeout value; conceptually a SET STATEMENT TIMEOUT seconds (this does not mean that we need to extend the Derby grammar; only the Network Server needs to understand this DRDA EXCSQLSET command). * In DRDAConnThread.parseEXCSQLSETobjects() on the server side, recognize the SET STATEMENT TIMEOUT text, parse the timeout value and remember it for the coming EXCSQLSTT command. Do NOT invoke executeUpdate() with the SET statement [see note below]. * In DRDAConnThread.parseEXCSQLSTT(), check if a timeout value has been set; if so, use it (by setting the timeout value on the server-side Statement object before calling execute/executeQuery). -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: New features for next release .... (Was: Grant and Revoke ... DERBY-464...)
Rick Hillegas wrote: I like Lance's suggestion and would like to propose it as a general policy. I think that this will handle Army's XML work as well as the other new 10.2 datatypes: If you add a new datatype to a server release (e.g., BOOLEAN or XML), then you must specify the following: Where would we specify these? In the code, in the documentation, in the functional spec, somewhere else, all of the above? By specify do you mean for informational/doc purposes, or are you referring to actual code in the server/client that does this specifying? 1) A legacy datatype to which the new type maps when the server talks to old clients. The legacy type is some datatype in JDBC 2.0 java.sql.Types. This is the type which old clients see in DatabaseMetaData and ResultSetMetaData. Does old client here simply mean Derby clients with Derby versions earlier than the server version (regardless of client-side jvm), or does it also mean old clients in the sense that you defined it in one of your other posts, namely: - The DB2JCC client - The 10.1 client - A 10.2 client running on jdk1.3 (JDBC 2), jdk1.4 (JDBC 3), or jdk1.5 (JDBC 3) (assuming the appropriate Derby and jvm versions are substituted in)? Army
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
I thought Rick's suggestion of adding the UNSIGNED keyword was a good solution -- we can get the best of both worlds... David Francois Orsini wrote: Since Sybase, MySQL and MS SQL Server have had support for UNSIGNED TINYINT for many years (at least for 2 of them), offering support for an UNSIGNED TINYINT rather than SIGNED at this point makes more sense and can only be good for Derby's adoption (and that a sufficient reason for adding it IMHO) (SIGNED TINYINT could always be enabled later _if_ required but JDBC does not require the type to be signed in the first place) - it brings value for getting Derby more adopted from users looking to migrate from other known and popular RDBMS (not just from the ones which got most market shares)...and as far as the footprint as previously mentioned, it is good to offer support for a 1-Byte datatype which does matter indeed when running in a small-device environment. --francois begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: [jira] Updated: (DERBY-704) Large page cache kills initial performance
There appear to be definite improvements here in terms of the consistency of throughput and CPU usage. Are the regular massive spikes in throughput and CPU usage related to the issues that Oystein was raising about checkpointing? Thanks, David Knut Anders Hatlen (JIRA) wrote: [ http://issues.apache.org/jira/browse/DERBY-704?page=all ] Knut Anders Hatlen updated DERBY-704: - Attachment: throughput.png cpu-usage.png Attached graphs showing the throughput and CPU usage running with and without the patch. The graphs show the average of seven runs, throughput and CPU usage were reported every 30 seconds. 12 clients were running an update-intensive load on the database. The database had 10 GB of user data, and the page cache size was 512 MB. The CPU usage is relatively low because the tests were run on an 8-CPU machine. The first 30 minutes the CPU usage was higher and the throughput significantly lower without the patch than with the patch. Large page cache kills initial performance -- Key: DERBY-704 URL: http://issues.apache.org/jira/browse/DERBY-704 Project: Derby Type: Bug Components: Services, Performance Versions: 10.1.2.2, 10.1.3.0, 10.2.0.0, 10.1.2.1, 10.1.2.0, 10.1.1.2, 10.1.1.1, 10.1.1.0 Environment: All platforms Reporter: Knut Anders Hatlen Assignee: Knut Anders Hatlen Fix For: 10.2.0.0 Attachments: DERBY-704.diff, cpu-usage.png, derbyall_report.txt, throughput.png When the page cache is large the performance gets lower as the page cache is being filled. As soon as the page cache is filled, the throughput increases. In the period with low performance, the CPU usage is high, and when the performance increases the CPU usage is lower. This behaviour is caused by the algorithm for finding free slots in the page cache. If there are invalid pages in the page cache, it will be scanned to find one of those pages. However, when multiple clients access the database, the invalid pages are often already taken. This means that the entire page cache will be scanned, but no free invalid page is found. Since the scan of the page cache is synchronized on the cache manager, all other threads that want to access the page cache have to wait. When the page cache is large, this will kill the performance. When the page cache is full, this is not a problem, as there will be no invalid pages. begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: Building jars from 10.1.2.1 source fails: Manifest is invalid
Monday, November 14, 2005, 7:33:51 PM CET, Andrew McIntyre wrote:
On Nov 14, 2005, at 9:00 AM, John Embretsen wrote:
has anyone tried to build the Derby jars based on the 10.1.2.1 (10.1.2
Release Candidate #2) source code archive? (We are supposed to be able
to do this, right?)
A google search led me to a thread in the mailing list archive of
Apache
Gump: http://tinyurl.com/742ln.
It seems that Andrew has come across the same build error fairly
recently, and that this is/was a subversion problem.
So, Andrew, any ideas why this error appears when I try to build the
jars based on the code from the 10.1.2.1 source archive?
Yes, there is a problem with svnversion in subversion 1.1.x and
earlier that causes an additional carriage return to be appended to
directories that are reported as 'exported', even when run with the -
n option. The property ${changenumber} which contains the version
reported by subversion then contains an additional carriage return,
which in turn breaks the manifest file format by setting the top-
level Sealed attribute apart from the rest of the top-level
attributes as you noted.
I checked in a fix to the trunk so that Gump could continue to run.
It's a rather simple patch, which you could apply to the build.xml
from the source distribution, attached to the end of this mail.
Excellent, I tried your patch (i.e., I added the striplinebreaks filter reader
to the top-level build.xml manually), and ant buildjars now works.
This problem has been fixed in Subversion 1.2.0 and above. Upgrading
to the latest subversion will also remedy the problem. For this
reason, I would consider it a non-showstopper for holding up the
10.1.2.1 release, but I will merge the fix over to the 10.1 branch
for future releases.
Yes, it would be unfortunate if this is considered a showstopper for the release
(still, it is too bad I didn't try this while the vote was still active), but I
have one question:
I am using Subversion 1.1.4, but this should not matter in this case, should it?
I mean, I guess it is the subversion version of the person doing the release
packaging (in this case Kathey, right?) that matters, or am I misunderstanding
something here?
--
John
andrew
Index: build.xml
===
--- build.xml (revision 330289)
+++ build.xml (revision 330290)
@@ -700,9 +700,13 @@
property name=derby.jar.topdir value=${basedir}/jars/
mkdir dir=${derby.jar.dir}/
mkdir dir=${derby.jar.dir}/lists/
- loadfile srcFile=${basedir}/changenumber.properties
+loadfile srcFile=${basedir}/changenumber.properties
failonerror=false
- property=changenumber/
+ property=changenumber
+ filterchain
+striplinebreaks/
+ /filterchain
+/loadfile
condition property=changenumber value=???
not
isset property=changenumber/
Re: Building jars from 10.1.2.1 source fails: Manifest is invalid
On Nov 14, 2005, at 11:52 AM, John Embretsen wrote: I am using Subversion 1.1.4, but this should not matter in this case, should it? I mean, I guess it is the subversion version of the person doing the release packaging (in this case Kathey, right?) that matters, or am I misunderstanding something here? No, it is the version of Subversion of the user running the buildjars target from the source distribution that matters, not the person packaging the release. The manifest is generated by Ant at the time that the buildjars target is run, not when the source distribution is created. So, in this case, yes, the version of Subversion that you are running is what makes the difference, and if you upgraded your Subversion to 1.2.0 or later, you would be able to run the buildjars target as well as if you had applied the patch that I sent. andrew
[jira] Commented: (DERBY-680) In ij, executing a prepared statement with numeric/decimal parameter fails with NullPointerException in J2ME/CDC/FP
[
http://issues.apache.org/jira/browse/DERBY-680?page=comments#action_12357621 ]
Deepa Remesh commented on DERBY-680:
By changing the above code to check for the value returned by ps.getMetaData()
, the prepared statements q10 and q11 listed in the description above work. The
test lang/cast.sql also passes with JSR169.
---
// In J2ME there is no object
that represents
// a DECIMAL value. By default
use String to
// pass values around, but for
integral types
// first convert to a integral
type from the DECIMAL
// because strings like 3.4 are
not convertible to
// an integral type.
ResultSetMetaData psmd =
ps.getMetaData();
if(psmd != null) {
switch
(psmd.getColumnType(c))
{
case Types.BIGINT:
ps.setLong(c,
rs.getLong(c));
break;
case Types.INTEGER:
case Types.SMALLINT:
case Types.TINYINT:
ps.setInt(c,
rs.getInt(c));
break;
default:
ps.setString(c,rs.getString(c));
break;
}
}
else {
ps.setString(c,rs.getString(c));
}
---
But the following case does not work because there is no way to get parameter
metadata in JSR169 since ParameterMetaData is not available. So, there is no
equivalent to switch (ps.getMetaData().getColumnType(c)) to determine the the
column/parameter type and call the corresponding setXXX method.
---
ij create table int_tab (i int);
ij prepare i1 as 'insert into int_tab values (?)';
ij execute i1 using 'values 3.4';
IJ WARNING: Autocommit may close using result set
ERROR 22018: Invalid character string format for type INTEGER.
ERROR 22018: Invalid character string format for type INTEGER.
at org.apache.derby.iapi.error.StandardException.newException(StandardEx
ception.java:311)
at org.apache.derby.iapi.types.DataType.invalidFormat(DataType.java:1049
)
at org.apache.derby.iapi.types.SQLInteger.setValue(SQLInteger.java:340)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.setString(EmbedPrep
aredStatement.java:452)
at org.apache.derby.impl.tools.ij.util.DisplayMulti(util.java:684)
at org.apache.derby.impl.tools.ij.utilMain.displayResult(utilMain.java:3
98)
at org.apache.derby.impl.tools.ij.utilMain.go(utilMain.java:290)
at org.apache.derby.impl.tools.ij.Main.go(Main.java:203)
at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:169)
at org.apache.derby.impl.tools.ij.Main.main(Main.java:75)
at org.apache.derby.tools.ij.main(ij.java:56)
---
I would like to know if anyone has suggestions to handle this. Thanks.
In ij, executing a prepared statement with numeric/decimal parameter fails
with NullPointerException in J2ME/CDC/FP
---
Key: DERBY-680
URL:
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
+1. Original argument of ease of migration from Sybase, Microsoft servers goes for a toss with a SIGNED implementation. There are too many issues with the current proposal. I am for making this follow closer to current implementations, if all. Satheesh Francois Orsini wrote: Since Sybase, MySQL and MS SQL Server have had support for UNSIGNED TINYINT for many years (at least for 2 of them), offering support for an UNSIGNED TINYINT rather than SIGNED at this point makes more sense and can only be good for Derby's adoption (and that a sufficient reason for adding it IMHO) (SIGNED TINYINT could always be enabled later _if_ required but JDBC does not require the type to be signed in the first place) - it brings value for getting Derby more adopted from users looking to migrate from other known and popular RDBMS (not just from the ones which got most market shares)...and as far as the footprint as previously mentioned, it is good to offer support for a 1-Byte datatype which does matter indeed when running in a small-device environment. --francois
Re: RowLocation lifetime
MM == Mike Matrigali [EMAIL PROTECTED] writes: MM Not really. MM I don't think even a unique key constraint meets the requirements, MM as the row can be deleted and a different row can be created. HeapRowLocation contains a RecordHandle which if I have understood this correctly refers to a RecordId in this case. Looking at the code, RecordId consists of pageId and a sequence number within each page. It seems to me that RecordIds should be unique and not reused. Could not this be used as a stable handle? MM Rick Hillegas wrote: Hi Mike, It appears that RowLocations are stable under some combinations of isolation levels and lock granularities but that, in general, a RowLocation isn't guaranteed to be stable even for the duration of a statement. Is there some other, more stable handle on a heap row? By more stable I mean something which uniquely identifies the row across statements, transactions, or connections? Thanks, -Rick -- Øystein
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
David W. Van Couvering wrote: I thought Rick's suggestion of adding the UNSIGNED keyword was a good solution -- we can get the best of both worlds... So more non-standard syntax? Why is it better for a SQL Server/Sybase application to change their types to TINYINT UNSIGNED, instead of SMALLINT? Or even SMALLINT with a check constraint to limit the range. Dan. David Francois Orsini wrote: Since Sybase, MySQL and MS SQL Server have had support for UNSIGNED TINYINT for many years (at least for 2 of them), offering support for an UNSIGNED TINYINT rather than SIGNED at this point makes more sense and can only be good for Derby's adoption (and that a sufficient reason for adding it IMHO) (SIGNED TINYINT could always be enabled later _if_ required but JDBC does not require the type to be signed in the first place) - it brings value for getting Derby more adopted from users looking to migrate from other known and popular RDBMS (not just from the ones which got most market shares)...and as far as the footprint as previously mentioned, it is good to offer support for a 1-Byte datatype which does matter indeed when running in a small-device environment. --francois
[jira] Commented: (DERBY-614) Execution failed because of a Distributed Protocol Error
[
http://issues.apache.org/jira/browse/DERBY-614?page=comments#action_12357630 ]
Francois Orsini commented on DERBY-614:
---
I have done a bit of reading on the subject (which was a great learning
exercise btw) and I find Brian's approach with 1) and 2) to be appropriate.
We'll need to ensure the remainder data attached to the resultset is
released/freed if the CLSQRY DDM command is sent by the requester or due to
some unexpected error condition (as part of ENDQRYRM) returned from the server
- but that should happen as part of resultset deallocation logic..
As far as handling MAXBLKEXT in 3) - I'd say that we obviously need to take it
into account since a client (requester) could set a value different than zero
(i.e. other DRDA clients out there).
Interestingly enough, the DDM specifications mention the following for
MAXBLKEXT:
A value of zero indicates that the requester is not capable of receiving extra
query blocks of answer set data.
which is the default we have right now, hence the server should not be
returning extra query blocks of QRYDTA...would that also be why the client sent
CLSQRY in the first place...I mean, if the client set a MAXBLKEXT of zero, then
it will not expect reading further data from the server after the initial reply
(just some thoughts).
Now, the specs also say the following about MAXBLKEXT:
A value of minus one indicates that the requester is capable of receiving the
entire result set. (am reading this as the requester will be able to receive
the entire resultset without any particular limit on the number of blocks of
answer set data returned by the server) - this is not very clear from the specs
so not completely sure about this.
Hence, should not we set '-1' as the default value of MAXBLKEXT for the Derby
DRDA networkclient driver instead of zero? Since and IMHO, the (derby) server
should by default try to return the entire resultset (in chunks/clocks as
appropriate) to the derby networkclient which has control as far as reading
more data or not (via CLSQRY) anyway.
Now, in case of MAXBLKEXT limit being reached (or in MAXBLKEXT value being
zero) then the server should probably terminate the query via a ENDQRYRM reply
and some error condition (Severity Code).
Execution failed because of a Distributed Protocol Error
Key: DERBY-614
URL: http://issues.apache.org/jira/browse/DERBY-614
Project: Derby
Type: Bug
Components: Network Server
Versions: 10.1.1.0
Environment: Linux 2.4, Sun JDK 1.4.2_07, full SYSINFO will be attached.
Reporter: Bryan Pendleton
Attachments: clientSideTrace.txt, clientStack.txt, derby-614.zip,
derbyTrace.txt, query.txt, serverSideTrace.txt, sysinfo.derby
I am intermittently receiving Distributed Protocol Error exceptions in my
client code. Simultaneously,
my derby.log is recording Distributed Protocol Error exceptions on the server
side.
I cannot reliably reproduce this problem.
However, the problem always occurs in the same section of my application, so
I can at least
describe that section of my application. (I'll do so in an attachment).
Here is the error I receive on the server side. Sometimes, the CODPNT is 2116
and the
Error Code Value is 1d, rather than 2114 and e.
2005-10-05 02:10:23.663 GMT Thread[DRDAConnThread_2,5,main] (DATABASE =
BuildFar
m), (DRDAID = GA0A0026.P7E6-4182154075488704215{136532}), Execution failed
becau
se of a Distributed Protocol Error: DRDA_Proto_SYNTAXRM; CODPNT arg = 2114;
Er
ror Code Value = e
Execution failed because of a Distributed Protocol Error:
DRDA_Proto_SYNTAXRM;
CODPNT arg = 2114; Error Code Value = e
2005-10-05 02:10:23.663 GMT Thread[DRDAConnThread_2,5,main] (DATABASE =
BuildFar
m), (DRDAID = GA0A0026.P7E6-4182154075488704215{136532}), null
null
org.apache.derby.impl.drda.DRDAProtocolException
at org.apache.derby.impl.drda.DRDAConnThread.throwSyntaxrm(Unknown
Sourc
e)
at org.apache.derby.impl.drda.DRDAConnThread.missingCodePoint(Unknown
So
urce)
at org.apache.derby.impl.drda.DRDAConnThread.parseCNTQRY(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.splitQRYDTA(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.writeFDODTA(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.writeQRYDTA(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Sou
rce)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: New features for next release .... (Was: Grant and Revoke ... DERBY-464...)
Daniel John Debrunner wrote: Rick Hillegas wrote: I like Lance's suggestion and would like to propose it as a general policy. I think that this will handle Army's XML work as well as the other new 10.2 datatypes: If you add a new datatype to a server release (e.g., BOOLEAN or XML), then you must specify the following: Maybe replace 'you must' with 'you can'. If someone has the itch to add a type for embedded only, then I don't think they should be forced to make it work with old or new clients. Also for something like XML datatype, is there any requirement to make it be available (in any form) with old clients? Is it not sufficient to say if you want to use the XML data type you must use the 10.2 client. If someone wants to do that work, that's fine, but I don't see it as a requiement. I'm concerned that if you can create a column, you ought to be able to poke values into it and then peek at them. In addition, some code has to go into the network layer, if only to raise exceptions when new datavalues try to leak across the wire. I suspect that detecting incompatibilities is the hard part. 1) A legacy datatype to which the new type maps when the server talks to old clients. The legacy type is some datatype in JDBC 2.0 java.sql.Types. This is the type which old clients see in DatabaseMetaData and ResultSetMetaData. Can old clients that are running as JDBC 3.0 see types from JDBC 3.0? I'm not sure I understand the question. Are you thinking about BOOLEAN and TINYINT? The Derby network layer seems to tightly couple JDBC type with transport format. Your question makes me think of another issue I have not addressed: What happens if a 10.2 client running at JDBC 3.0 selects an XML value? I am struggling to describe datatype behavior without a matrix of release and vm levels, which would confuse product support. Here's another attempt to summarize the behavior: 1) The new datatype has an associated legacy 2.0 datatype for use with old clients and old JDBC levels. 2) The new datatype's server level is the Derby release which introduces the datatype. Similarly, the new datatype's JDBC level is the JDBC version which introduced the type. 3) To see the new datatype, the client must run at or above the datatype's server and JDBC levels. 4) Otherwise, the client sees the legacy datatype. I'm not sure that's simpler than a matrix, but there it is. :) Is this just for the network server, how about embedded, e.g. running Derby on JDK 1.3/1.4? Thanks for raising this case. Let's keep it simple and apply the same rules. Fortunately, in the embedded case the client runs at the server's rev level. So it's just a question of JDBC level. So let's imagine a 10.2 embedded server running at JDBC 3.0. The customer creates BOOLEAN columns and peeks/pokes them as BOOLEAN. The customer create SQLXML columns but peeks/pokes them as CLOB. 2) A pair of ResultSet.get() and PreparedStatement.set() methods which old clients can use to access the new datavalues. These must be get() and set() methods which appear in JDBC 2.0 ResultSet and PreparedStatement. They should be the get() and set() methods most natural to the legacy datatype in (1). These methods determine how the datavalues flow across DRDA. Just curious as to why specifying the getXXX and setXXX method is required, doesn't it follow from the legacy JDBC type specified? Or is there some deeper thought here I am missing? For example, in your example, with NCLOB can the client use setClob, setString etc? Nothing deep, I'm just being pedantic. As you note, the mapping determines the legacy dataype and datavalue and therefore the transport format. The customer should be able to use any getXXX and setXXX method that works for that transport format. We could leave this as an exercise for the reader. Dan.
Re: New features for next release .... (Was: Grant and Revoke ... DERBY-464...)
Rick Hillegas wrote: Daniel John Debrunner wrote: Maybe replace 'you must' with 'you can'. If someone has the itch to add a type for embedded only, then I don't think they should be forced to make it work with old or new clients. I'm concerned that if you can create a column, you ought to be able to poke values into it and then peek at them. That's exactly it. I'm concerned, that's your itch. :-) Dan.
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
Daniel John Debrunner wrote: David W. Van Couvering wrote: I thought Rick's suggestion of adding the UNSIGNED keyword was a good solution -- we can get the best of both worlds... So more non-standard syntax? Why is it better for a SQL Server/Sybase application to change their types to TINYINT UNSIGNED, instead of SMALLINT? Some customers may not care if they have to double the column size. Others may care. Or even SMALLINT with a check constraint to limit the range. Dan. You can also implement NOT NULL as a check contraint. Most databases consider that to be an inefficiency which it's worth optimizing with custom logic.
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
Rick Hillegas wrote: You can also implement NOT NULL as a check contraint. Most databases consider that to be an inefficiency which it's worth optimizing with custom logic. Custom logic that's defined by the SQL Standard. :-) Dan.
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
On 11/14/05, Rick Hillegas [EMAIL PROTECTED] wrote: Daniel John Debrunner wrote:David W. Van Couvering wrote:I thought Rick's suggestion of adding the UNSIGNED keyword was a goodsolution -- we can get the best of both worlds... So more non-standard syntax? Why is it better for a SQL Server/Sybaseapplication to change their types to TINYINT UNSIGNED, instead ofSMALLINT?Some customers may not care if they have to double the column size. Others may care. Dan's argument which is mine too I believe is in respect with users migrating from Sybase/MS SQL Server apps using TINYINT to Derby - if we provide an unsigned type by default then they don't have anything to change in respect with that type (same semantics) - MySQL has support for both SIGNED and UNSIGNED so what not have (unsigned) TINYINT supported in Derby by default and encompass a wider range of databases supporting (unsigned) TINYINT which in the end will ease migration and help Derby's adoption... Or even SMALLINT with a check constraint to limit the range.Dan. You can also implement NOT NULL as a check contraint. Most databasesconsider that to be an inefficiency which it's worth optimizing withcustom logic.
Open Source Database technical collaboration mailing list
At the Open Source Database Conference (http://www.opendbcon.net/) in Frankfurt last week, MySQL AB hosted a dinner for the database speakers and others involved in open source databases. The idea was to see how open source databases could collaborate. Øystein Grøvlen and myself were at the dinner, along with folks from MySQL, Postgres, Firebird, Hsqldb and SQLLite. One idea that came up was a public, archived mailing list where technical discussions could occur. This list is now active at: http://lists.osdbconsortium.org/mailman/listinfo/hackers As Derby developers you may want to subscribe. The other main idea was for an official open source database consortium, though I think that got lost in the exact purpose of such a group. Issues arose around what would being a member mean and should commercial databases companies be excluded, which naturally would have excluded almost everyone at the table from joining. :-) Dan.
Re: [jira] Commented: (DERBY-699) Allow schema and savepoint names starting with SYS
Bernt M. Johnsen wrote: Satheesh Bandaram commented on DERBY-699: - Derby needs to treat many schemas starting with SYS.. as system schemas... like SYSCAT, SYS, SYSFUN, SYSSTAT, SYSPROC, SYSIBM, SYSCS_DIAG, SYSCS_UTIL. It may be possible to add future system schemas too, like SYSFUN was added recently. Any change to allow user SYS schemas needs to be made very carefully. I agree that Derby needs to reserve a part of the namespace for future system schema names. But I find it unreasonable to reserve all names starting with SYS because SYSTEM is a very common word and that this would disallow schema names like SYSTEM_DIAGNOSTICS, SYSTEM_STATUS etc. A common prefix like this might be a hurdle for porting existing schemas and applications to Derby (that's the way I discovered the restriction). I also see from the comments in the code that this restriction was added as a DB2-compliance, and think that we should revert this restriction. Do other databases have restrictions in creating schemas? The best approach would probably be to add a non-standard character to the identifiers so that a non-standard prefix could be used. E.g. SYS$. That's pretty ugly in my mind, not sure why would would force such use on Derby's users. If we went down this path, then we have a somewhat unique current prefix for some newer system schemas, SYSCS_. Or something like SYSDERBY_ could be picked. Dan.
[jira] Resolved: (DERBY-704) Large page cache kills initial performance
[ http://issues.apache.org/jira/browse/DERBY-704?page=all ] David Van Couvering resolved DERBY-704: --- Resolution: Fixed Checked in patch, revision number 344270 bash-2.05$ svn commit Sendingjava/engine/org/apache/derby/impl/services/cache/Clock.java Transmitting file data . Committed revision 344270. Large page cache kills initial performance -- Key: DERBY-704 URL: http://issues.apache.org/jira/browse/DERBY-704 Project: Derby Type: Bug Components: Services, Performance Versions: 10.1.1.0, 10.2.0.0, 10.1.2.0, 10.1.1.1, 10.1.1.2, 10.1.2.1, 10.1.3.0, 10.1.2.2 Environment: All platforms Reporter: Knut Anders Hatlen Assignee: Knut Anders Hatlen Fix For: 10.2.0.0 Attachments: DERBY-704.diff, cpu-usage.png, derbyall_report.txt, throughput.png When the page cache is large the performance gets lower as the page cache is being filled. As soon as the page cache is filled, the throughput increases. In the period with low performance, the CPU usage is high, and when the performance increases the CPU usage is lower. This behaviour is caused by the algorithm for finding free slots in the page cache. If there are invalid pages in the page cache, it will be scanned to find one of those pages. However, when multiple clients access the database, the invalid pages are often already taken. This means that the entire page cache will be scanned, but no free invalid page is found. Since the scan of the page cache is synchronized on the cache manager, all other threads that want to access the page cache have to wait. When the page cache is large, this will kill the performance. When the page cache is full, this is not a problem, as there will be no invalid pages. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (DERBY-689) Various improvements to compatibility test.
[ http://issues.apache.org/jira/browse/DERBY-689?page=all ] Rick Hillegas updated DERBY-689: Attachment: bug689_2.diff bug689_2.diff further improves this suite by adding a target for running just one combination of client/clientVM/server/serverVM. The target is called oneCombination and it is explained by -projecthelp. Since this patch does not touch any code involved in the product or the regression tests, it should be safe to check in as is: M java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\testScript.xml A java\testing\org\apache\derbyTesting\functionTests\tests\junitTests\compatibility\oneCombo.properties Various improvements to compatibility test. --- Key: DERBY-689 URL: http://issues.apache.org/jira/browse/DERBY-689 Project: Derby Type: Improvement Reporter: Rick Hillegas Assignee: Rick Hillegas Attachments: bug689.diff, bug689_2.diff Improve compatibility test as various reviewers requested. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
Hi, Francois == Francois Orsini [EMAIL PROTECTED] wrote: Francois Dan's argument which is mine too I believe is in respect with users Francois migrating from Sybase/MS SQL Server apps using TINYINT to Derby - if we Francois provide an unsigned type by default then they don't have anything to change Francois in respect with that type (same semantics) - MySQL has support for both Francois SIGNED and UNSIGNED so what not have (unsigned) TINYINT supported in Derby Francois by default and encompass a wider range of databases supporting (unsigned) Francois TINYINT which in the end will ease migration and help Derby's adoption... So it would seem this boils down to which consideration is more important; semantic consistency (=TINYINT is by default signed, since other integer types are signed in Derby), or ease of portability to Derby (= TINYINT is unsigned by default, since 2 major players use that). I would argue that a single byte data type is useful in its own right, and will favor signed semantics as the default - to keep things clean, SQL is ugly enough as it is.. If we want the unsigned version also, I would vote for the UNSIGNED keyword, the type is non-standard syntax in SQL anyway. Disclaimer: I guess I should add I am a little biased against unsigned integer types in general, I have seen far too many bugs due to indiscriminate mixing of signed and unsigned types over the years ;-) Thanks, Dag
Re: [jira] Commented: (DERBY-695) Re-enable the TINYINT datatype
This is highly arguable - you say SQL is ugly as it is (which is arguable by itself ;)) but then you think it's ok to add a non-standard UNSIGNED keyword if we want the unsigned version which has been there for more than 15 years at least in very well known RDBMS out there ;) Either way is fine really (as long both semantics end-up being supported at the end)...It's a question of views and priorities from different people having different niches ;) --francois If we end-up supporting both semantics, On 11/14/05, Dag H. Wanvik [EMAIL PROTECTED] wrote: Hi, Francois == Francois Orsini [EMAIL PROTECTED] wrote:Francois Dan's argument which is mine too I believe is in respect with users Francois migrating from Sybase/MS SQL Server apps using TINYINT to Derby - if weFrancois provide an unsigned type by default then they don't have anything to changeFrancois in respect with that type (same semantics) - MySQL has support for both Francois SIGNED and UNSIGNED so what not have (unsigned) TINYINT supported in DerbyFrancois by default and encompass a wider range of databases supporting (unsigned)Francois TINYINT which in the end will ease migration and help Derby's adoption... So it would seem this boils down to which consideration is moreimportant; semantic consistency (=TINYINT is by default signed, sinceother integer types are signed in Derby), or ease of portability to Derby (= TINYINT is unsigned by default, since 2 major players usethat).I would argue that a single byte data type is useful in its own right,and will favor signed semantics as the default - to keep things clean, SQL is ugly enough as it is..If we want the unsigned versionalso, I would vote for the UNSIGNED keyword, the type is non-standardsyntax in SQL anyway.Disclaimer: I guess I should add I am a little biased against unsigned integer types in general, I have seen far too many bugs due toindiscriminate mixing of signed and unsigned types over the years ;-)Thanks,Dag
[jira] Resolved: (DERBY-682) Add tests for error codes for severe exceptions
[ http://issues.apache.org/jira/browse/DERBY-682?page=all ] Deepa Remesh resolved DERBY-682: Fix Version: 10.2.0.0 Resolution: Fixed Resolved by svn revision 332645 Add tests for error codes for severe exceptions --- Key: DERBY-682 URL: http://issues.apache.org/jira/browse/DERBY-682 Project: Derby Type: Bug Components: Test Versions: 10.2.0.0 Reporter: Deepa Remesh Assignee: Deepa Remesh Priority: Minor Fix For: 10.2.0.0 Attachments: derby-682.diff, derby-682.status Add tests to test error codes for disconnect exceptions and other severe errors for client and embedded drivers. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (DERBY-682) Add tests for error codes for severe exceptions
[ http://issues.apache.org/jira/browse/DERBY-682?page=all ] Deepa Remesh closed DERBY-682: -- Add tests for error codes for severe exceptions --- Key: DERBY-682 URL: http://issues.apache.org/jira/browse/DERBY-682 Project: Derby Type: Bug Components: Test Versions: 10.2.0.0 Reporter: Deepa Remesh Assignee: Deepa Remesh Priority: Minor Fix For: 10.2.0.0 Attachments: derby-682.diff, derby-682.status Add tests to test error codes for disconnect exceptions and other severe errors for client and embedded drivers. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-587) Providing JDBC 4.0 support for derby
[ http://issues.apache.org/jira/browse/DERBY-587?page=comments#action_12357651 ] Satheesh Bandaram commented on DERBY-587: - Thanks, Narayanan, for the follow up. I think only the ICLA is the pending issue now. Jean had mentioned last time that either your name show on the list or get some kind of acknowledgement that Apache received your fax. I will commit your changes just after that. Jean, let us know if there is any other way. I hate to be holding up good and important work, but don't know what else I could do. Thanks to Rick for the reviews and for running the tests. Providing JDBC 4.0 support for derby Key: DERBY-587 URL: http://issues.apache.org/jira/browse/DERBY-587 Project: Derby Type: New Feature Components: JDBC Versions: 10.2.0.0 Reporter: V.Narayanan Assignee: V.Narayanan Priority: Minor Fix For: 10.2.0.0 Attachments: jdbc4.0.sxw, jdbc4.diff -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (DERBY-280) Wrong result from select when aliasing to same name as used in group by
[ http://issues.apache.org/jira/browse/DERBY-280?page=comments#action_12357654 ] Satheesh Bandaram commented on DERBY-280: - Rick, what have you decided about this patch? Would you like me to commit this? As I mentioned, I believe, this patch improves on the current situation, though doesn't address the underlying issue. If you want me to commit the patch, please do update the patch with latest code base... sqlgrammar has changed. Wrong result from select when aliasing to same name as used in group by --- Key: DERBY-280 URL: http://issues.apache.org/jira/browse/DERBY-280 Project: Derby Type: Bug Components: SQL Reporter: Bernt M. Johnsen Assignee: Rick Hillegas Priority: Minor Attachments: bug280.diff Wrong result from select when aliasing to same name as used in group by. Example: If we have the following table: ij select * from tt; I |J --- 1 |2 2 |3 1 |2 2 |3 2 |3 5 rows selected The following select is ok: ij select i, count(*) as cnt from tt group by i; I |CNT --- 1 |2 2 |3 2 rows selected But this one returns wrong result in the aliased column: ij select i, count(*) as i from tt group by i; I |I --- 1 |1 2 |2 2 rows selected -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] Commented: (DERBY-506) Implement Statement.setQueryTimeout in the Client Driver
Satheesh Bandaram (JIRA) wrote:
2) What would happen if a 10.2 client tries to send a SET STATEMENT_TIMEOUT
statement to a 10.1 server? There doesn't seem to be any mechanism to prevent
unpleasant events.
I was reviewing the patch and decided to look into your question more, and I
think there is actually code there to catch this. The choice of a SET ...
syntax was a good one in this particular case: there is already code in
DRDAConnThread that looks for SET ... statements that are included as part of
an EXCSQLSET command, and if the statement isn't recognized, the server will
simply return a warning and ignore it. The code that does this is in the
parseEXCSQLSETobjects() method:
...
try {
drdaStmt.getStatement().executeUpdate(sqlStmt);
} catch (SQLException e) {
// if this is a syntax error, then we take it
// to mean that the given SET statement is not
// recognized; take note (so we can throw a
// warning later), but don't interfere otherwise.
if (e.getSQLState().equals(SYNTAX_ERR))
hadUnrecognizedStmt = true;
else
// something else; assume it's serious.
throw e;
}
break;
...
// Now that we've processed all SET statements (assuming no
// severe exceptions), check for warnings and, if we had any,
// note this in the SQLCARD reply object (but DON'T cause the
// EXCSQLSET statement to fail).
if (hadUnrecognizedStmt) {
SQLWarning warn = new SQLWarning(One or more SET statements +
not recognized., 01000);
throw warn;
} // end if.
...
To confirm, I tried to run jdbcapi/SetQueryTimeoutTest from a client that had
the patch applied against a server that did not. The result was the following
diff, which looks correct to me:
Test SetQueryTimeoutTest FAILED
org.apache.derbyTesting.functionTests.tests.jdbcapi.SetQueryTimeoutTest$TestFailedException:
fetch did not time out. Highest execution time: 7000 ms
In other words, the call to SET QUERY_TIMEOUT failed with a syntax error and
was ignored, and thus the statement never timed out.
Does that sound correct?
Army
[jira] Updated: (DERBY-239) Need a online backup feature that does not block update operations when online backup is in progress.
[ http://issues.apache.org/jira/browse/DERBY-239?page=all ] Suresh Thalamati updated DERBY-239: --- Attachment: onlinebackup_3.diff This patch adds code to support real-time online backup with unlogged operations. A consistent backup can not be made if there are pending transactions with unlogged operations or if unlogged operations occur when backup is in progress. Because container files can be copied to the backup before the transaction is committed and the data pages are flushed as part of the commit. As there is no transaction log for unlogged operations, while restoring from the backup database can not be restored to a consistent state. To make a consistent online backup in this scenario, this patch: 1) blocks online backup until all the transactions with unlogged operation are committed/aborted. 2) implicitly converts all unlogged operations to logged mode for the duration of the online backup, if they are started when backup is in progress. This patch also adds a test to test the online backup in parallel with some DML, DDL and unlogged operations. TESTS : derbyall test suite passed on Windows XP/JDK142 It would be great if some can review and commit this patch. svn stat: M java\engine\org\apache\derby\impl\store\raw\xact\Xact.java M java\engine\org\apache\derby\impl\store\raw\xact\XactFactory.java M java\engine\org\apache\derby\impl\store\raw\RawStore.java M java\engine\org\apache\derby\impl\store\raw\data\BaseDataFileFactory.java M java\engine\org\apache\derby\iapi\store\raw\xact\RawTransaction.java M java\engine\org\apache\derby\iapi\store\raw\xact\TransactionFactory.java M java\testing\org\apache\derbyTesting\functionTests\tests\storetests\st_1.sql A java\testing\org\apache\derbyTesting\functionTests\tests\store\OnlineBackupTest1_app.properties A java\testing\org\apache\derbyTesting\functionTests\tests\store\OnlineBackup.java M java\testing\org\apache\derbyTesting\functionTests\tests\store\copyfiles.ant A java\testing\org\apache\derbyTesting\functionTests\tests\store\OnlineBackupTest1.java M java\testing\org\apache\derbyTesting\functionTests\master\st_1.out A java\testing\org\apache\derbyTesting\functionTests\master\OnlineBackupTest1.out M java\testing\org\apache\derbyTesting\functionTests\suites\storemore.runall Need a online backup feature that does not block update operations when online backup is in progress. Key: DERBY-239 URL: http://issues.apache.org/jira/browse/DERBY-239 Project: Derby Type: New Feature Components: Store Versions: 10.1.1.0 Reporter: Suresh Thalamati Assignee: Suresh Thalamati Attachments: onlinebackup.html, onlinebackup_1.diff, onlinebackup_2.diff, onlinebackup_3.diff Currently Derby allows users to perfoms online backups using SYSCS_UTIL.SYSCS_BACKUP_DATABASE() procedure, but while the backup is in progress, update operations are temporarily blocked, but read operations can still proceed. Blocking update operations can be real issue specifically in client server environments, because user requests will be blocked for a long time if a backup is in the progress on the server. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
