Re: Interfacing with embedded derby from an outside process/shell

[Kristian Waagan]

No, that is not supported using the embedded driver only.

You either have to shut down the application, or connect using the network
driver (while the application itself connects using the embedded driver).
The "embedded server mode" is described at [1] and in the admin guide.

Do you need the application to be up while you interface with the database
manually?

Regards,
-- 
Kristian

[1] https://stackoverflow.com/q/374833/4473053

Den ons. 2. aug. 2017, 16:46 skrev Dmitry Buzolin <dbuz5ga...@gmail.com>:

> Thanks you Kristian,
>
> Booting/opening is clear. But say Derby is booted/opened by an
> application, can I connect to the same database via IJ tools and interface
> with the database (UPDATE/INSERT/DELETE)? Or this is not supported?
>
> On Aug 2, 2017, at 10:39 AM, Kristian Waagan <krist...@apache.org> wrote:
>
> Hi Dmitry,
>
> Only one process can open / boot a specific database (usually a set of
> files on disk) at a given time.
> Depending on your requirements you can:
>  a) shut down the application and boot the database using any standard
> JDBC tool (using the embedded driver)
>  b) if you need to be able to access the database while the application is
> running, you need to enable the Derby network server mode and connect to it
> using the network driver. This can be done in more than one way, for
> instance by specifying a system property on startup. This may have security
> implications for your application.
>
> If this is a permanent requirement, it should be possible to start and
> stop the network server from within your application (i.e. by adding
> support for this feature in your application). See the Derby Administration
> guide for more info.
>
>
> Hope this helps,
> --
> Kristian
>
> ons. 2. aug. 2017 kl. 14:47 skrev Dmitry Buzolin <dbuz5ga...@gmail.com>:
>
>> Hi List!
>>
>> I have a tough problem: My application runs Derby in embedded mode and I
>> have a requirement to change/correct data inside this database.
>> What is a best way to do this if it is possible? For example if I open
>> this database location via "file://patch to db folder” will this work
>> properly wrt data consistency, locking etc (since the database is accessed
>> from within the app code)?
>
>
>

Re: Interfacing with embedded derby from an outside process/shell

[Kristian Waagan]

Hi Dmitry,

Only one process can open / boot a specific database (usually a set of
files on disk) at a given time.
Depending on your requirements you can:
 a) shut down the application and boot the database using any standard JDBC
tool (using the embedded driver)
 b) if you need to be able to access the database while the application is
running, you need to enable the Derby network server mode and connect to it
using the network driver. This can be done in more than one way, for
instance by specifying a system property on startup. This may have security
implications for your application.

If this is a permanent requirement, it should be possible to start and stop
the network server from within your application (i.e. by adding support for
this feature in your application). See the Derby Administration guide for
more info.


Hope this helps,
-- 
Kristian

ons. 2. aug. 2017 kl. 14:47 skrev Dmitry Buzolin :

> Hi List!
>
> I have a tough problem: My application runs Derby in embedded mode and I
> have a requirement to change/correct data inside this database.
> What is a best way to do this if it is possible? For example if I open
> this database location via "file://patch to db folder” will this work
> properly wrt data consistency, locking etc (since the database is accessed
> from within the app code)?

Re: Write Cache - Data Corruption

[Kristian Waagan]

Correct, you absolutely don't need a UPS for Derby to function. The issues
to consider with an abrupt shutdown would be loss of uncommitted
transactions and recovery time.

There's also the usual additional risk that something goes awry in the OS
or with the hardware itself. That risk is likely way lower than the risk of
programming or operator error. In any case, this is when you want your
backup and restore procedures to be tested and working.

As Rick pointed out, and which is described on the webpage referred to, a
UPS can give you power of higher quality and protect your hardware.

Regards,
-- 
Kristian

Den søn. 16. jul. 2017, 16:42 skrev Shreyans Jain :

> Why would we need any kind of power backup if every thing is directly
> written to disk, write cache being off. Only risk would be of losing last
> transaction data.
>
> On Jul 16, 2017 7:58 PM, "Rick Hillegas"  wrote:
>
>> On 7/15/17 10:12 PM, Shreyans Jain wrote:
>>
>> If we have write cache off, do we need a ups or some other power backup ?
>>
>> I'm not an expert on ups's, but my understanding is that they provide
>> other kinds of protective power conditioning which you would want in a
>> production environment. See, for instance,
>> https://en.wikipedia.org/wiki/Uninterruptible_power_supply.
>>
>> Hope this helps,
>> -Rick
>>
>>
>> On Sat, Jul 15, 2017 at 8:02 PM, Rick Hillegas 
>> wrote:
>>
>>> On 7/14/17 6:21 PM, Shreyans Jain wrote:
>>>
 If we have a ups and we properly shutdown the OS, will then we have to
 disable write cache for the hard-disk in order to prevent data corruption.
 Are there any chances left for the data to be corrupted by power failure
 with write cache on?

 Regards,
 Shreyans Jain

>>>
>>> That will reduce your risk. But operating systems can crash before
>>> flushing the cache to disk.
>>>
>>>
>>
>>

Re: Derby Database Corruption Issues

[Kristian Waagan]

Hi Shreyans,

I believe this is still valid advice, but the details depend on your
IO-system, your operating system and your server hardware.

The issue here is that Derby is told the data has been written to stable
storage, but then it has in fact not. If the machine crashes at this point
data may be lost.
For instance, if you are using a SAN there is typically a battery / UPS
backed buffer (or other types of buffers that don't loose data in case of a
power outage) on the "storage side", but for this to be useful you must
ensure data is pushed through the IO driver and is not buffered locally by
the operating system.

Using asynchronous writes is a typical way of obtaining better [write]
performance. In many cases the downside is a higher chance of data
corruption.
This may or may not be acceptable for your application.


Regards,
-- 
Kristian

Den ons. 12. jul. 2017, 20:04 skrev Shreyans Jain :

> I was reading https://wiki.apache.org/db-derby/DatabaseCorruption and i
> found prevention of corruption which states
> "Switch off the machine's write caching" . now the article is written in
> 2013. Is it still application to derby database corruption or now it has no
> effect.
>
> Regards,
> Shreyans Jain
>

Re: how to retrieve a number with more dec places

[Kristian Waagan]

Hi Bob,

Assuming you have stored it correctly, have you tried using getBigDecimal?


-- 
Kristian

Den tir. 28. feb. 2017, 07:03 skrev Bob M :

Hi

I have a derby database with records that contain a price e.g. 1.21098

I retrieve it by the following line

price = rs.getDouble("price");

when I print it out I get :- 1.21

What do I need to change so that I retrieve the full price with all its
decimal places?

Bob M



--
View this message in context: http://apache-database.10148.
n7.nabble.com/how-to-retrieve-a-number-with-more-dec-places-tp147257.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: Export with column names in first line

[Kristian Waagan]

tor. 8. sep. 2016 kl. 07.05 skrev Eugster Christian BB_SK <
christian.eugs...@stadt.sg.ch>:

> I am afraid, I am not capable to open an issue. I do not know how I can
> achieve this. I cannot find a menu entry to open a new issue.
>

Hi Christian,

You'll probably need to sign up to get an account first (click "Log in").
After you log in, the "Create" button should show up in a prominent red
color.

It would be nice to log an improvement request and get it implemented, I
just want to mention that many administration and query tools can do the
kind of export you're looking for. This can be helpful if this is an ad-hoc
activity.


Regards,
-- 
Kristian

Thanks
>
> Christian
>
> -Ursprüngliche Nachricht-
> Von: Bryan Pendleton [mailto:bpendleton.de...@gmail.com]
> Gesendet: Donnerstag, 8. September 2016 01:54
> An: Derby Discussion
> Betreff: Re: Export with column names in first line
>
> > is there an opportunity to export a table with column names in first
> line of the exported file?
>
> I don't believe that is currently possible, no.
>
> It seems like a nice enhancement to suggest.
>
> I see some enhancement requests for SYSCS_EXPORT_TABLE logged as:
>
> https://issues.apache.org/jira/browse/DERBY-6850
>
> but yours seems rather a different idea than those.
>
> If you'd like to record your enhancement request, it seems to me you could
> either add it to DERBY-6850 or you could log a new suggestion.
>
> See: http://db.apache.org/derby/DerbyBugGuidelines.html for more details
> about how Derby tracks suggestions and bug reports.
>
> thanks,
>
> bryan
>
>
>
> 
>
> Hinweis: Diese Mitteilung ist ausschliesslich für die als Adressaten
> bezeichneten Personen bestimmt. Sie kann vertrauliche und rechtlich
> geschützte Informationen enthalten. Sollten Sie diese Mitteilung irrtümlich
> erhalten haben, bitten wir Sie, uns unverzüglich zu benachrichtigen und
> diese Mitteilung zu löschen. Das unerlaubte Kopieren sowie die unbefugte
> Weitergabe dieses E-Mails ist nicht gestattet. Besten Dank.
>

Re: What is the fastest query to check for an existence of a row

[Kristian Waagan]

Den 25.05.2016 23.12, skrev Bergquist, Brett:
> I have a table with 80,640,704 rows.   My query currently is:
> 
>  
> 
> SELECT
> 
> COUNT(*) AS USE_COUNT
> 
> FROM
> PKG_9145E_V1.COSEDDROPPROFILEDSCPTABLEBUNDLE_COSEDDROPPROFILEDSCPTABLEENTRY
> CBCE
> 
> WHERE CBCE.COSEDDROPPROFILEDSCPTABLEENTRY_ID = 768
> 
> AND CBCE.COSEDDROPPROFILEDSCPTABLEBUNDLE_ID != 2
> 
>  
> 
> This query is going to return a count of 78,569.   
> 
>  
> 
> What I really want to know is if there is any row that satisfies the
> condition.   I have logic that needs to run if there is such a row.   So
> is there a more efficient query in Derby that will short circuit when it
> find a row?

Hi Brett,

FETCH/OFFSET [1] comes to mind, i.e. going from

SELECT COUNT(*)
FROM CBCE
WHERE ...

to

SELECT some_column_or_a constant
FROM CBCE
WHERE ...
FETCH FIRST ROW ONLY


Since there's no ORDER BY (or other relevant operations / clauses) in
your query, that should hopefully cause Derby to stop processing and
return once the first matching row is found.
I say hopefully, because I haven't verified it :)


Regards,
-- 
Kristian


[1] https://db.apache.org/derby/docs/10.12/ref/rrefsqljoffsetfetch.html

> 
>  
> 
> Thanks
> 
>  
> 
> Brett
> 
> 
> 
> Canoga Perkins
> 20600 Prairie Street
> Chatsworth, CA 91311
> (818) 718-6300
> 
> This e-mail and any attached document(s) is confidential and is intended
> only for the review of the party to whom it is addressed. If you have
> received this transmission in error, please notify the sender
> immediately and discard the original message and any attachment(s).

Re: Performance issue with same index name in multiple schemas

[Kristian Waagan]

Den 05.02.2016 13.22, skrev kosurusekhar:
> Hi All,
> 
> We are having three schemas with almost same table structure, indexes,
> queries, stored procedures in one database. But in one schema we are facing
> some performance issue. We are having same index names and table names in
> all schemas. In another schemas it is running without any problem. In one
> schema with basic load only application is becoming slow. 

Hi Sekhar,

Are the schemas approximately the same size in terms of data inserted
into the database?
Also, are the access patterns / loads roughtly the same?

Otherwise I'd follow some of Brett's pieces of advice. If you can
identiy the slow query (and the tables / indexes involved), you can
compare the plans for the performant schema and the slow schema.


Regards,
-- 
Kristian

> 
> Is there any relationship with table names and index names with multiple
> schemas? What ever we have queries/stored procedure's with performance
> issues, these queries/stored procedure's running with out any issue in
> another schema's. I am not understanding what is going wrong.
> 
> I could see some times single transaction is locking couple of tables more
> than once and not releasing. At this point of time we are feeling the
> slowness. 
> 
> Please help me with some points to sort out this situation.
> 
> Thanks in advance.
> 
> Regards
> Sekhar.
> 
> 
> 
> --
> View this message in context: 
> http://apache-database.10148.n7.nabble.com/Performance-issue-with-same-index-name-in-multiple-schemas-tp145507.html
> Sent from the Apache Derby Users mailing list archive at Nabble.com.
> 

Re: got wrong data with using '%' as condition

[Kristian Waagan]

Hi Li,

'_' is a wildcard character matching a single character. To search for the
literal '_' you have to escape it.

There may be several ways to do that, but one is described here:
https://db.apache.org/derby/docs/10.11/ref/rrefjdbc28456.html

Regards,
-- 
Kristian

tor. 9. jul. 2015, 18.56 skrev yiran9937 yiran9...@sina.com:

 Hi, I am Li. I started to learn to use the Derby no long time. I used
 derby database in Eclipse and start connection with ij tool which in
 derbytools.jar. The derby version is 10.9. Once, I select data from a table
 with SQL select * from tig_users where user_id like 'test_01_%';, but got
 below data. Note the first record, the user_id 'test_01@yiranlocal'
 should not matching with 'test_01_%', but I got it. I have no idea with
 this. I try to test this with the newest version 10.11, but got the same
 result. What happened and how can I get correct data result? I will very
 pleased for any suggestion. Thanks and Regards, Li
 --
 View this message in context: got wrong data with using '%' as condition
 http://apache-database.10148.n7.nabble.com/got-wrong-data-with-using-as-condition-tp144340.html
 Sent from the Apache Derby Users mailing list archive
 http://apache-database.10148.n7.nabble.com/Apache-Derby-Users-f95095.html
 at Nabble.com.

Re: Performance degradation with Derby Embedded under Tomcat vs Jetty

[Kristian Waagan]

And, by testing with a network server I think you can access JMX mbeans
with VisualVM/JConsole to check connection counts.

-- 
Kristian
13. apr. 2015 13:12 skrev Kristian Waagan krist...@apache.org:

 Hi Øivind,

 Have you turned on SQL-logging/-tracing in Derby to check the activity?

 Possible things to check:
 o connection IDs (distinct/count)
 o expensive connection validation queries (not all of these are direct
 SQL, check pool docs/conf)
 o commits?
 o different queries?

 While one would expect the load / queries to be the same, one never
 knows...

 In theory one would think the client driver was more affected by a pool
 misconfiguration.

 Regards,
 --
 Kristian
 13. apr. 2015 11:56 skrev Øyvind Harboe oyvind.har...@zylin.com:

 I've made them identical near as I can. It isn't the same
 implementation of the connection pool.

 Is it only EmbeddedDriver that would be negatively affected by a
 mis-configured connection pool?

 Is there some logging in Derby that I could enable to verify that
 things are set up correctly, i.e. that new connections are not made
 all the time?



 On Mon, Apr 13, 2015 at 11:16 AM, Brett Wooldridge
 brett.wooldri...@gmail.com wrote:
  Are you sure that under Tomcat the application is really using the
 Tomcat
  connection pool?  If not that would certainly account for the
 differences.
  Even if Tomcat DBCP is configured, checking that the pool
 configurations are
  similar would seem like an important check.
 
  -Brett
 
 
  On Mon, Apr 13, 2015 at 4:56 PM, Øyvind Harboe oyvind.har...@zylin.com
 
  wrote:
 
  I know the Tomcat setup isn't using the clientdriver when it slows
  down, because when I first tried to switch to the ClientDriver it
  failed. After I copied the derbyclient.xxx.jar to tomcat/lib/, it
  worked.
 
 
 
 
 
  On Mon, Apr 13, 2015 at 9:51 AM, Dyre Tjeldvoll
  dyre.tjeldv...@oracle.com wrote:
   On 04/13/2015 12:27 AM, Øyvind Harboe wrote:
  
   I found another crucial clue:
  
   if I use ClientDriver instead of EmbeddedDriver and connect to the
   Derby database running under Jetty, I get identical performance with
   Tomcat and Jetty.
  
   This is very strong indication that there is something about the
   combination of EmbeddedDriver and Tomcat that is gumming up the
 works.
  
  
   Could it be that your Tomcat setup always uses the ClientDriver for
 some
   reason? I don't know much about setting up Databases with AppServers,
   but
   based on the number of questions on SO and other places, it isn't
   trivial...
  
   --
   Regards,
  
   Dyre
 
 
 
  --
  Øyvind Harboe - Can Zylin Consulting help on your project?
  http://www.zylin.com/
 
 



 --
 Øyvind Harboe - Can Zylin Consulting help on your project?
 http://www.zylin.com/

Re: Performance degradation with Derby Embedded under Tomcat vs Jetty

[Kristian Waagan]

Hi Øivind,

Have you turned on SQL-logging/-tracing in Derby to check the activity?

Possible things to check:
o connection IDs (distinct/count)
o expensive connection validation queries (not all of these are direct SQL,
check pool docs/conf)
o commits?
o different queries?

While one would expect the load / queries to be the same, one never knows...

In theory one would think the client driver was more affected by a pool
misconfiguration.

Regards,
-- 
Kristian
13. apr. 2015 11:56 skrev Øyvind Harboe oyvind.har...@zylin.com:

 I've made them identical near as I can. It isn't the same
 implementation of the connection pool.

 Is it only EmbeddedDriver that would be negatively affected by a
 mis-configured connection pool?

 Is there some logging in Derby that I could enable to verify that
 things are set up correctly, i.e. that new connections are not made
 all the time?



 On Mon, Apr 13, 2015 at 11:16 AM, Brett Wooldridge
 brett.wooldri...@gmail.com wrote:
  Are you sure that under Tomcat the application is really using the Tomcat
  connection pool?  If not that would certainly account for the
 differences.
  Even if Tomcat DBCP is configured, checking that the pool configurations
 are
  similar would seem like an important check.
 
  -Brett
 
 
  On Mon, Apr 13, 2015 at 4:56 PM, Øyvind Harboe oyvind.har...@zylin.com
  wrote:
 
  I know the Tomcat setup isn't using the clientdriver when it slows
  down, because when I first tried to switch to the ClientDriver it
  failed. After I copied the derbyclient.xxx.jar to tomcat/lib/, it
  worked.
 
 
 
 
 
  On Mon, Apr 13, 2015 at 9:51 AM, Dyre Tjeldvoll
  dyre.tjeldv...@oracle.com wrote:
   On 04/13/2015 12:27 AM, Øyvind Harboe wrote:
  
   I found another crucial clue:
  
   if I use ClientDriver instead of EmbeddedDriver and connect to the
   Derby database running under Jetty, I get identical performance with
   Tomcat and Jetty.
  
   This is very strong indication that there is something about the
   combination of EmbeddedDriver and Tomcat that is gumming up the
 works.
  
  
   Could it be that your Tomcat setup always uses the ClientDriver for
 some
   reason? I don't know much about setting up Databases with AppServers,
   but
   based on the number of questions on SO and other places, it isn't
   trivial...
  
   --
   Regards,
  
   Dyre
 
 
 
  --
  Øyvind Harboe - Can Zylin Consulting help on your project?
  http://www.zylin.com/
 
 



 --
 Øyvind Harboe - Can Zylin Consulting help on your project?
 http://www.zylin.com/

Re: RawStoreDaemon thread in Derby Embedded Database

[Kristian Waagan]

Den 25.03.15 07.48, skrev Vivek Choudhary:
 Thanks Rick, Mike for the information provided about the Daemon thread. 
 It clears some understanding about the significance of these threads.
 
 In my legacy application it performs following operation - 
 1. writes data to table under jta transaction boundary (handled by 
 Hibernate)
 2. scheduled cron job runs in every night to delete old data
 
 Now, the issue which we are facing is the daemon thread is consuming 
 100% cpu usage and as result we have to restart our server to get this 
 clear out. After the server restart everything becomes normal.
 
 Any pointer what would be causing daemon thread to consumes cpu usage?

Hi Vivek,

Are you able to obtain some stack traces from the application when it's
using 100% CPU?


If it is indeed the RawStoreDaemon that's consuming the CPU, the stack
traces might give some clues. Post only the traces for the daemon thread
of you like.

Also, what's the version of your Derby installation?


Regards,
-- 
Kristian

 
 Thanks.
 Vivek
 
 

Re: Star Derby as a service

[Kristian Waagan]

Den 23.09.14 13:37, skrev Alessandro Manzoni:
 On Derby wiki pages
 (http://wiki.apache.org/db-derby/DerbyWindowsService) I found how to
 install Derby as a service.
 It's made by Tanuki wrapper, but I installed succesfully myself the
 service using Apache commons Daemon
 (http://commons.apache.org/proper/commons-daemon/index.html).
 Are there some particular reasons not to use commons Daemon?


Hi Alessandro,

Not that I'm aware of, just two different ways to achive the same thing.

Feel free to add a section on how to accomplish the task with the Apache
Commons Daemon.


Regards,
-- 
Kristian

Re: CPU Utilization

[Kristian Waagan]

On 07.01.2013 20:51, Pavel Bortnovskiy wrote:


Hello:

This is more of a general question. Our application uses Derby in the 
in-memory mode only.


When the application is configured to use complex queries, such 
configuration causes CPU utilization (on the Linux server) to go as 
high as 300% or even higher. Using simpler queries don't seem to lead 
to such high utilization. Is there any way to control or lower CPU 
utilization by Derby?




Hi Pavel,

I don't believe there is such a mechanism in place, at least not 
specifically for the in-memory back end.


Since this is in-memory, the latency associated with page operations is 
very low. Do you know if Derby is using the CPU for something useful 
(i.e. processing queries effectively), or is the CPU spent on wasteful 
activities?
One potential wasteful activity is moving data back and forth from the 
page cache (page cache too small?).

Is this a highly concurrent scenario?


Regards,
--
Kristian


Thank you,
Pavel.



Jefferies archives and monitors outgoing and incoming
e-mail. The contents of this email, including any
attachments, are confidential to the ordinary user of the
email address to which it was addressed. If you are not
the addressee of this email you may not copy, forward,
disclose or otherwise use it or any part of it in any form
whatsoever. This email may be produced at the request of
regulators or in connection with civil litigation.
Jefferies accepts no liability for any errors or omissions
arising as a result of transmission. Use by other than
intended recipients is prohibited. In the United Kingdom,
Jefferies operates as Jefferies International Limited;
registered in England: no. 1978621; registered office:
Vintners Place, 68 Upper Thames Street, London EC4V 3BJ.
Jefferies International Limited is authorised and
regulated by the Financial Services Authority.

Re: what do errors like these mean?

[Kristian Waagan]

On 21.12.2012 16:18, Pavel Bortnovskiy wrote:

Thank you, Knut, for your prompt response.

It seems that my caching of Prepared Statements is causing some problems.
In some previous responses, it was indicated that Derby is caching them 
internally anyway, so maybe a better approach for me is not to cache them on my 
side and create them anew? Most of my inserts/updates are done in batches, so I 
could create a PrepStmt before the batch and remove a reference to it at the 
end of the batch's execution. If the performance penalty for compilation of 
PrepStmt is not that great, then such approach may be more desirable to avoid 
errors in the production environment.


Hi Pavel,

Are you using the embedded driver or the client driver?
The client driver can cache statements on the client side if you use the 
ConnectionPoolDataSource. This may save you some round-trips, but note 
that this caching is in addition to the caching that happens on the 
server side.



--
Kristian



Thanks,
P.

-Original Message-
From: Knut Anders Hatlen [mailto:knut.hat...@oracle.com]
Sent: Friday, December 21, 2012 8:59 AM
To: Derby Discussion
Subject: Re: what do errors like these mean?

Pavel Bortnovskiy pbortnovs...@jefferies.com writes:


Once in a while, I see the following errors. What may cause them?

java.sql.SQLException: Container 1,329 not found.

The error means that one of the database files (table or index) cannot be 
found. It typically happens because some DDL operation (for example DROP INDEX, 
TRUNCATE TABLE or SYSCS_COMPRESS_TABLE) has removed the file, and an already 
compiled statement still references it.

The error indicates a bug in Derby, so if you find a way to reproduce it, or 
some pattern that seems to increase the likelihood of the error, please file a 
bug report.

Derby should invalidate already compiled statement referencing the table when 
DDL is performed on it, and that should make the statement recompile 
automatically the next time it is executed. There have been bugs in that area, 
though. (We fixed some of them in 10.9.1.0, in case you haven't already 
upgraded.)

One possible workaround in that case is to call the 
SYSCS_UTIL.SYSCS_EMPTY_STATEMENT_CACHE procedure to remove the stale query 
plans from the statement cache.

--
Knut Anders

Jefferies archives and monitors outgoing and incoming e-mail. The contents of 
this email, including any attachments, are confidential to the ordinary user of 
the email address to which it was addressed. If you are not the addressee of 
this email you may not copy, forward, disclose or otherwise use it or any part 
of it in any form whatsoever. This email may be produced at the request of 
regulators or in connection with civil litigation. Jefferies accepts no 
liability for any errors or omissions arising as a result of transmission. Use 
by other than intended recipients is prohibited. In the United Kingdom, 
Jefferies operates as Jefferies International Limited; registered in England: 
no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London 
EC4V 3BJ. Jefferies International Limited is authorised and regulated by the 
Financial Services Authority.

Re: unable to take backup after derby upgrade

[Kristian Waagan]

On 02.10.2012 17:59, Tomcat Programmer wrote:
I was able to backup my derby database successfully using the previous 
version 10.8.2.2  and the following command, running from ij:


ij CALL SYSCS_UTIL.SYSCS_BACKUP_DATABASE('/usr/local/derby10/backups');


After upgrading to 10.9.1.0 the same exact command fails with the 
following error:


ERROR 38000: The exception 'java.security.AccessControlException: 
Access denied (java.io.FilePermission /usr/local/derby10/backups/PRAT 
write)' was thrown while evaluating an expression.ERROR XJ001: Java 
exception: 'Access denied (java.io.FilePermission 
/usr/local/derby10/backups/PRAT write): 
java.security.AccessControlException'.


It may also help if you post the stack trace relevant to Derby (i.e. you 
don't need to include anything from your own code).



Regards,
--
Kristian




I am using the exact same copy of the java JRE, no changes.  After 
upgrading the derby version, the update process was run on the 
database and did not return any errors. No changes have been made to 
the filesystem.  The directory /usr/local/derby10/ as well as all of 
its child directories are owned by the same user id, which is used to 
run the database and do the backup. The file permissions are setup so 
this user id has full control of all files and directories.  The 
operating system is 32-bit linux.


What has changed in this release which is causing this problem?   How 
do I fix it?

Re: Need derby 10.0.2.1-bin.zip version

[Kristian Waagan]

On 28.09.2012 15:17, lakshmi Suresh wrote:

Dear Team ,

I am trying to set up a database for the CFEveryWhere environment. 
When i try to run the command

java org.apache.derby.drda.NetworkServerControl start


i am getting error as
*
could not load main class org.apache.derby.drda.networkservercontrol

*please advise me what needs to be done to make the connection up.*
*


Hi,

This looks like a classpath issue. Are you sure you have derbynet.jar on 
the classpath?



Regards,
--
Kristian


*

source :http://coldfusion.sys-con.com/node/48655


*

Re: Derby server listening on different interfaces

[Kristian Waagan]

On 19.09.2012 15:13, Maison Mo wrote:

Hello Derby users,

I have a rather unusual use case, where I have two clients using the same Derby 
DB. Thus I use a Derby server.
One client is on the same host as the derby server, whereas the second one is 
not ; I have security concerns with the non local client that will most 
probably require SSL with client/server authentication.
For performance reasons, I planned to use localhost interface without ssl for 
the localhost client, and ssl authentication for the remote client only.

I'd like to know if this is feasible with Derby ? Based on my readings on admin 
documentation, I am afraid not.
What would you recommend in that case ?


Hello,

Have you considered if it is feasible to have the local client use the 
embedded driver and the remote client use the network driver?
This setup is referred to as embedded server. There are many things 
that may make it hard to use, for instance that your local client is 
shut down but the remote client most still be able to access the database.


I haven't confirmed this, but I too suspect that listening to one 
interface using SSL and one without SSL isn't supported by Derby.
Given that the embedded server approach is unfeasible, you may have to 
choose between improved security wrt the remote client and overhead for 
the local client :(



Regards,
--
Kristian



Regards,

   M. Maison

Re: derby db not found

[Kristian Waagan]

On 04.09.2012 00:33, Rainier Diaz Ferrer wrote:
Hello Derby team, my name is Rainier, i'm having problems trying to 
connect to my derby database using network server. I set the derby 
home directory by using this. System.setproperty(derby.system.home, 
mylocalpathtoderbydb); this code line goes right before starting the 
server. When i try to connect it throws a database not found 
exception, if i copy the database to the current directory, where my 
application is executing from, it works fine, but i want it to find my 
database somewhere else,  which is in mylocalpathtoderbydb, set as 
derby.system.home. i would appreciate an answer. Thank you. Rainier


Hi Rainier,

Depending on how the driver is loaded, the property may be set too late.
You can try to set the property as a system property on the command 
line, or to specify it in derby.properties in your current working 
directory to verify that theory.


Another option is to use an absolute path to you database, but that's 
often not what you want.



Regards,
--
Kristian

Re: All Derby Downloads Returning 404 from All Mirrors

[Kristian Waagan]

On 26.08.2012 17:05, Michael Urban wrote:
This has been going on for at least 14 hours now. All Derby downloads 
are returning 404 errors from ALL mirrors, including the backups. I 
looked at the directory structure of the Web server and it doesn't 
even appear there is a directory for where the derby downloads should be.


Hi Michael,

Thanks for letting us know.

I'm not having any issues with downloading Derby.
Do you have an example URL which is still broken?

That said, the mirror selection script doesn't seem to work correctly 
for me, so I'm wondering if something has been changed recently.



Regards,
--
Kristian



Needless to say, this does not inspire a lot of confidence in the 
Derby project when all links to download the software are broken for 
more than 14 hours, and no one seems to notice, or if they do notice, 
they don't care enough to fix it on a weekend.

Re: All mirrors returning 404

[Kristian Waagan]

On 26.08.2012 03:09, Equ Zephyr wrote:

All mirrors the latest releases of Derby are returning 404 errors.


Hi Equ,

Thanks for letting us know.

I'm not having any issues with downloading Derby.
Do you have an example URL which is still broken?

That said, the mirror selection script doesn't seem to work correctly 
for me, so I'm wondering if something has been changed recently.



Regards,
--
Kristian

Re: jOOQ Tutorial contribution for the Blogs and articles about Derby page

[Kristian Waagan]

On 01.08.2012 14:11, Rick Hillegas wrote:

On 8/1/12 1:17 AM, Lukas Eder wrote:

Hello,

I was wondering if Derby would be interested in a jOOQ tutorial
contribution that could be put here:
http://db.apache.org/derby/blogs/index.html#persistence

I have done this before for H2:
http://www.h2database.com/html/tutorial.html#using_jooq

A more thorough tutorial can be found on the jOOQ website, for MySQL:
http://www.jooq.org/tutorial.php

I'd offer to contribute and maintain this tutorial, should there be
any relevant changes on the jOOQ side.
Please, let me know what you think!

Cheers
Lukas


Hi Lukas,

This sounds like a great idea. I'd be happy to add your jOOQ tutorial 
to the Derby website at the location you suggested.


Thanks a lot for the offer, Lukas :)

Rick, do you intend to add a link to somewhere, or to host the tutorial 
on the Derby site?

I'd prefer the latter, but that poses some hurdles:
 a) someone (TM) may have to format the content as DITA
 b) Lukas will have to rely on others to push updates, or earn his 
commit privileges :)


(b) may not be much of a problem once the initial page has been completed.
I'd be willing to help with (a) if that is required.

Lukas, on another note, I've noticed that you've run into trouble 
because Derby doesn't have a bunch of built-in functions [1]. Have you 
identified other areas where Derby integration is painful?



Regards,
--
Kristian

[1] I believe the reason is that [most of] these functions are not 
described in the SQL standard(s), and standards compliance is on Derby's 
charter.




Thanks,
-Rick

Re: Derby 10.9.1.0 Incompatible with Substituted Drives on Windows

[Kristian Waagan]

On 31.07.2012 12:32, John I. Moore, Jr. wrote:


Derby Users and Developers,

I have an application that has used Derby (with the Network Server) 
for years, and overall I have been extremely pleased.However, I ran 
into a problem when trying to upgrade to 10.9.1.0.


I have three computing environments that I work in, which can be 
thought of a development, test, and production.For reasons not worth 
the trouble of explaining, these three environments are slightly 
different, but until the latest release of Derby I could make them 
appear to be very similar by the use of the Windows subst command, 
which associates a path with a drive letter.For example, the command 
subst K: C:\SomeDir makes the system appear to have a K drive, 
which in reality is simply mapped to the directory SomeDir on the C 
drive.All previous versions of Derby handled databases on this 
substituted K drive without any problems, but version 10.9.1.0 breaks 
on it.I was able to get around this problem by setting up a virtual 
drive using a different approach, so now I have been able to upgrade 
Derby, but I wanted to point out the problem is case others have 
encountered it.




Hi John,

Do you have any more information about what breaks?
I.e., are you able to boot the database?
Do you get an error message?
On which versions of Windows have you observed the problem?


Thanks,
--
Kristian


John Moore

Re: AW: schema-questions

[Kristian Waagan]

On 23.07.2012 15:34, malte.kem...@de.equens.com wrote:

Actually I found 2 hints

1.
use SET CURRENT SCHEMA
2.
Make your own custom authentication



For completeness, one can also specify the schema explicitly in the 
queries of course. I don't know if there are scenarios where this 
approach won't work.



--
Kristian

Re: AW: AW: AW: schema-questions

[Kristian Waagan]

On 19.07.2012 18:44, Tim Watts wrote:

My impression is that your goal is to explore web app development using
Derby.  In general, focusing on performance tuning in the absence of an
actual performance issue is not going to be very productive.


The above sounds like a good piece of advice to me.

That said, in the case of using Derby from an application server (single 
server, one connection pool) I would restrict the number of concurrent 
connections by configuring the connection pool.
Without having tested the timeslice and max threads properties of the 
network server in depth, my impression is that you should not use them 
unless you really have to. One such scenario may be if you for some 
reason allow direct access to your network server and have no other 
mechanism to restrict the number of concurrent connections.


With regards to Tomcat deployment, do you know where the derby.log file 
ends up?

I've seen it end up in the logs-directory of the Tomcat installation.


Regards,
--
Kristian

Re: Strange OutOfMemory in unittest

[Kristian Waagan]

On 28.06.2012 12:18, mogoye wrote:

Problem is found :

You were right, the problem was that my JVM starts with only 64m of heap
size.
I don't know why but I was sure I read somewhere that with JDK 1.5 the
default max heap size was 512m...


It can vary depending on the machine, platform, Java version and whether 
you run with -client or -server. What's happening has changed with the 
Java versions, so this is getting a bit muddy...

It may be useful to set the maximum heap size explicitly.

In any case, good to hear that you figured out what the problem was :)


Cheers,
--
Kristian


So I was on the wrong track.

Sorry to make you loose precious time for a stupid mistake

Re: Strange OutOfMemory in unittest

[Kristian Waagan]

On 27.06.12 15:15, mogoye wrote:


Hello,

I'm currently working on a some junit tests to validate My DB using Derby
1.8.1.2.
My code is using JPA annotations and Eclipselink implementation.

The purpose of this test is to evaluate the performances of my requests.
My test is doing the following things :
- Create an empty DB.
- Fill DB with 200 000 entries
- Performs some requests.
- Check answers are Ok and measure time to proccess them.

The thing I don't understant is why my test can be executed 50% of the time
with no problem, and the other 50% I have OutOfMemory that occurs at various
tests execution.

I don't understand why the same code executed few seconds later can sometime
be a success or failure.


For example here is one of the request that is executed.
In that case the request is a native querry and a very basic request:
SELECT * FROM TICKET ORDER BY EXPECTED_DATE ASC OFFSET 33000 ROWS FETCH
FIRST 20 ROWS ONLY


My DB (measured on the file system) is only 100Mb so not very big one
See bellow the stack trace.

I have no idea how I can go further as I'm not very familliar with derby
tools or logs to help debugging.

Does any one have some idea ?


As already mentioned, you should check what your maximum heap size is 
set to. If Derby is run with the default page cache size, it normally 
shouldn't occupy much more than 4 MB of heap.


If your VM supports it, you could dump the heap when the OOME occurs 
[1], or use something like VisualVM (maybe with the visualgc plugin) to 
observe the memory usage. If you obtain a dump, you have to extract 
information from it afterwards, for instance with jhat. If you search 
the web, you'll probably find other tools doing similar things.
Note that you can always obtain heap dumps with VisualVM, and 
information like the max heap size.


You could also sample the heap manually by using a tool like jmap to get 
a heap histogram.


In many cases the thread stack trace is a red herring, at least in 
multi-threaded applications, since the OOME can happen at any place 
where memory is needed/allocated.



Regards,
--
Kristian

[1] For the Sun/Oracle JVMs that would be -XX:+HeapDumpOnOutOfMemoryError



Thanks a lot.
Sylvain




Here is the complete Stack trace I can get (sometimes I do not have any
stack info):

java.lang.OutOfMemoryError: Java heap space
at
sun.text.resources.DateFormatZoneData.getContents(DateFormatZoneData.java:299)
at
sun.text.resources.DateFormatZoneData.loadLookup(DateFormatZoneData.java:144)
at
sun.text.resources.DateFormatZoneData.getKeys(DateFormatZoneData.java:100)
at
sun.text.resources.DateFormatZoneData.getKeys(DateFormatZoneData.java:105)
at 
java.text.DateFormatSymbols.loadZoneStrings(DateFormatSymbols.java:449)
at 
java.text.DateFormatSymbols.initializeData(DateFormatSymbols.java:489)
at java.text.DateFormatSymbols.init(DateFormatSymbols.java:103)
at java.util.TimeZone.retrieveDisplayNames(TimeZone.java:412)
at java.util.TimeZone.getDisplayNames(TimeZone.java:401)
at java.util.TimeZone.getDisplayName(TimeZone.java:352)
at java.util.Date.toString(Date.java:1024)
at java.lang.String.valueOf(String.java:2615)
at java.lang.StringBuffer.append(StringBuffer.java:220)
at
org.apache.derby.impl.services.stream.BasicGetLogHeader.getHeader(Unknown
Source)
at org.apache.derby.iapi.error.ErrorStringBuilder.appendln(Unknown 
Source)
at
org.apache.derby.iapi.services.context.ContextManager.cleanupOnError(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
Source)
at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeQuery(Unknown
Source)
at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
at
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeSelect(DatabaseAccessor.java:931)
at
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:607)
at
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:535)
at
org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:1702)
at
org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:566)
at

Re: Conglomerate could not be created?

[Kristian Waagan]

On 19.06.2012 09:45, John English wrote:

On 05/06/2012 18:45, Dag H. Wanvik wrote:


Looks like a bug to me. The error XSCH4 Conglomerate could not be
created isn't being tested for in the Derby tests as far as I can
see. I also searched the bug tracker
(https://issues.apache.org/jira/browse/DERBY) in vain for this
particular error.

Looking in the code where the error is reported in the source code, it
looks like it shouldn't happen. Could you post a JIRA bug report for
this error, please?  It you are able to make a stand-alone repro that
will increase your chances of having it fixed.


After trying to boil this down and after much head-scratching, I've
realised that I do the insert using a method in a DB utility class.
There are two methods: dbInsert() and dbUpdate(). The difference is
that doInsert() uses Statement.RETURN_GENERATED_KEYS when creating
the prepared statement, and then calls stat.getGeneratedKeys() after
the insertion.

My problem seems to arise because I used doInsert to execute the insert
even though there isn't a generated key. If I use dbUpdate() instead,
the problem goes away.

My attempt at a standalone repro tries to mimic this behaviour like so:

  AUTOCOMMIT OFF;

  INSERT INTO resources ...;
  INSERT INTO resource_usage ...;
  PREPARE p AS ...;
  EXECUTE p USING ...;
  VALUES IDENTITY_VAL_LOCAL();

  COMMIT;

but this works. So I suspect that IDENTITY_VAL_LOCAL is implemented
differently, and the problem is due to Statement.RETURN_GENERATED_KEYS
being used with a statement that doesn't generate a key.

The stack trace I posted earlier began:
java.sql.SQLException: Conglomerate could not be created.
  at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown 
Source)
  at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown 
Source)
  at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
Source)
  at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown 
Source)
  at 
org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown 
Source)
  at 
org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown 
Source)
  at 
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown 
Source)
  at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown 
Source)
  at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source) 


  at utils.DataManager.dbInsert(DataManager.java:1056)

dbInsert lines 1056/1057 look like this:
  stat.executeUpdate();
  ResultSet r = stat.getGeneratedKeys();
so the failure happens when the prepared statement is executed, just
before the call to getGeneratedKeys().

Does this suggest a cause for the Conglomerate could not be created
error to anyone?


Hi John,

I believe your assumptions are correct.
This is a bug in Derby where the temporary row holder, used to hold 
generated keys, overflows and a temporary backing conglomerate is 
attempted created with a zero-length row template.


The bug can easily be reproduced using JDBC. Unless someone beats me to 
it, I'll log a JIRA and attach a regression test for the bug.



Regards,
--
Kristian



TIA,

Re: Conglomerate could not be created?

[Kristian Waagan]

On 20.06.2012 13:05, John English wrote:

On 20/06/2012 11:45, Kristian Waagan wrote:

Hi John,

I believe your assumptions are correct.
This is a bug in Derby where the temporary row holder, used to hold
generated keys, overflows and a temporary backing conglomerate is
attempted created with a zero-length row template.

The bug can easily be reproduced using JDBC. Unless someone beats me to
it, I'll log a JIRA and attach a regression test for the bug.


Excellent, many thanks. I tried a JDBC version of my IJ script but it
still didn't reproduce it, so if you've managed to do it that's great.


Logged as DERBY-5823 [1].

The bug is data dependent in that it requires a certain number of rows 
to be inserted, so that may be what's causing your repro to fail 
triggering the bug.



--
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-5823

Re: DERBY-5818

[Kristian Waagan]

On 14.06.2012 14:12, SAURABH KEJRIWAL wrote:
I want the version details through SQL query. Could you please help me 
for this?


Hi Saurabh,

Can you just confirm that doing this via a JDBC method call isn't an 
option for you?
If JDBC is okay, maybe the method 
DatabaseMetaData.getDatabaseProductVersion gives you what you need.



--
Kristian

Re: Conglomerate could not be created?

[Kristian Waagan]

On 04.06.12 14:52, Rick Hillegas wrote:

On 6/2/12 3:18 AM, John English wrote:

On 01/06/2012 15:19, John English wrote:

I'm having trouble with the following error: Conglomerate could not be
created. It happens when I do this:

INSERT INTO resource_usage (resid,itemid,itemtype)
(SELECT resid,?,? FROM resource_usage
WHERE itemid=?
AND itemtype=?
AND NOT EXISTS (SELECT resid FROM resource_usage
WHERE itemid=? AND itemtype=?));


I also get the same error from this, which looks fairly innocuous to me:
INSERT INTO resource_usage (resid,itemid,itemtype)
(SELECT resid,20,'final' FROM resource_usage WHERE itemid=? AND
itemtype=?);
Is it perhaps because I'm inserting into the same table I'm selecting
from?

Hi John,

Do you have a stack trace for this error?


My extremely simple attempt at reproducing this didn't succeed on trunk 
(10.10), and I could duplicate the matching rows (only changing the 
itemtype column).
Do you also have the schema and example of data that triggers the 
problem? Are there many matching rows for the select?



Thanks,
--
Kristian



Thanks
-Rick

--
John English If we were to define a religion to be a system of thought
which contains unprovable statements, so it contains an element of
faith, then Gödel has taught us that not only is mathematics a
religion but it is the only religion able to prove itself to be one.
— John Barrow, /Pi in the Sky /

Re: Derby Locks - best practices

[Kristian Waagan]

On 01.06.2012 21:50, Pavel Bortnovskiy wrote:

Hello, David, thanks for your quick response.

Usually it's one thread per in-memory table. Tables can be updated at random times and 
their random rows may be updated, some rows deleted or new rows inserted. In some other 
configuration, to avoid deletions, updates and inserts, the in-memory table is truncated and then 
all the records (the new state of the source data) are inserted into it.


For clarity, are you referring to using Derby's memory subprotocol when 
talking about in-memory tables? As an example, that would be 
'jdbc:derby:memory:mydb;create=true', as opposed to the on-disk version 
that would be 'jdbc:derby:mydb;create=true'.
In terms of locking there is nothing special about in-memory database in 
Derby, except for the likely event that some operations may be faster 
in-memory than on-disk (which could affect timing, but many other things 
can do that too).


Two common pieces of advice when it comes to locking is to reduce the 
duration of the locks, and to reduce the scope/granularity of the locks.
There may also be  application specific considerations to take, like 
acceptable isolation levels, access patterns and schema design.


In general your application should be prepared to handle lock timeouts, 
whereas deadlocks indicate that the access/lock patterns of your 
application need to be improved.



--
Kristian



The thread which runs SQL against all those tables frequently may do a scan of 
the whole table.

-Original Message-
From: David Zanter [mailto:dzan...@gmail.com]
Sent: Friday, June 01, 2012 3:46 PM
To: Derby Discussion
Subject: Re: Derby Locks - best practices

Do mean the scenario of:
 Multiple threads are updating the exact same rows

or
 Multiple threads doing updates to different rows, but due to 
queries/indexes/etc are causing contention between each other.

On Fri, Jun 1, 2012 at 3:16 PM, Pavel Bortnovskiypbortnovs...@jefferies.com  
wrote:

Hello, all:



Derby is used in my application in the in-memory only mode. For a long
time Derby's lock logic caused no worries, but recently some use cases
failed with lock timeouts. Thus I'm looking for guidance on best
practices for handling locks in Derby. A use-case which may cause
timeouts to obtain a
lock: one thread is executing an SQL statement which accesses two (or
more) in-memory tables. Those two tables are being modified by  other
threads at random times. So, situations in which the SQL is executed
for a long time and the other threads are frequently updating the
tables may cause lock timeouts.



Besides best practices to avoid timeouts and deadlocks, I would like
to ask the following questions:

1)  What's the default length of lock timeouts?

2)  Does my app need another layer of synchronization
mechanism/locks to avoid attempts to update in-memory tables or execute SQLs 
against them?

3)  Can my application utilize Derby's locks through some API - to
query their state or to use them in making a decision of whether to
batch updates or to execute them, to wait or execute the SQLs?



Your help would be greatly appreciated,



Pavel.

Jefferies archives and monitors outgoing and incoming e-mail. The
contents of this email, including any attachments, are confidential to
the ordinary user of the email address to which it was addressed. If
you are not the addressee of this email you may not copy, forward,
disclose or otherwise use it or any part of it in any form whatsoever.
This email may be produced at the request of regulators or in
connection with civil litigation. Jefferies accepts no liability for
any errors or omissions arising as a result of transmission. Use by
other than intended recipients is prohibited. In the United Kingdom,
Jefferies operates as Jefferies International Limited; registered in
England: no. 1978621; registered office: Vintners Place, 68 Upper
Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised 
and regulated by the Financial Services Authority.

Jefferies archives and monitors outgoing and incoming e-mail. The contents of 
this email, including any attachments, are confidential to the ordinary user of 
the email address to which it was addressed. If you are not the addressee of 
this email you may not copy, forward, disclose or otherwise use it or any part 
of it in any form whatsoever. This email may be produced at the request of 
regulators or in connection with civil litigation. Jefferies accepts no 
liability for any errors or omissions arising as a result of transmission. Use 
by other than intended recipients is prohibited. In the United Kingdom, 
Jefferies operates as Jefferies International Limited; registered in England: 
no. 1978621; registered office: Vintners Place, 68 Upper Thames Street, London 
EC4V 3BJ. Jefferies International Limited is authorised and regulated by the 
Financial Services Authority.

Re: limit on the number of columns

[Kristian Waagan]

On 24.05.2012 23:33, Patrick Meyer wrote:

That would be excellent! I think it would be a great feature to have in
Derby.


Hi Patrick,

Can you say anything about how many columns would be needed to support 
these use-cases?

Are we talking about a few thousand, ten thousand, or even more columns?


Regards,
--
Kristian



Patrick

-Original Message-
From: Rick Hillegas [mailto:rick.hille...@oracle.com]
Sent: Thursday, May 24, 2012 12:08 PM
To: derby-user@db.apache.org
Subject: Re: limit on the number of columns

On 5/24/12 4:26 AM, Patrick Meyer wrote:

I am aware that Derby has a limit of 1,012 columns for each table, but
many users of my application (it is a program for statistical
analysis) have very large files that go well beyond this number of
columns. Does anyone know of a strategy for using multiple tables to
present one large virtual table to end users? Is there a way to
chain tables together to have an endless number of columns? Is this
something that can be done through SQL statements? Any advice,
examples or documentation on such a strategy would be greatly
appreciated.

Thanks,

Patrick


Hi Patrick,

This appears to me to be an arbitrary limit in Derby, one which we could
investigate lifting. To track this issue, I have filed
https://issues.apache.org/jira/browse/DERBY-5781. This kind of change would
have to appear in a feature release. The 1012 limit also applies to the
number of columns in a SELECT list. This is another arbitrary limit which we
should consider lifting:
https://issues.apache.org/jira/browse/DERBY-5782

Thanks,
-Rick

Re: Not able to run the derby server on Eclipse

[Kristian Waagan]

On 21.05.12 10:30, Mahesh Sitaram Patil11 wrote:

Hello Team,

I have configured the derby database for my Java application in Eclipse
but I am not able to start the Derby server in eclipse. Please find the
below screen shot of the consol error I am getting:

Please let me know the solution for the above issue. Appreciate your help.


Hi,

I'm not familiar with Eclipse, but to me it looks like there may be 
several versions of Derby on the classpath. I can't read the whole error 
message, but the relevant method was changed at one point (its return 
value changed from Object to void).



Regards,
--
Kristian



Thanks,

-Mahesh Patil

-9960955960

 CAUTION - Disclaimer *
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are 
not
to copy, disclose, or distribute this e-mail or its contents to any other 
person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has 
taken
every reasonable precaution to minimize this risk, but is not liable for any 
damage
you may sustain as a result of any virus in this e-mail. You should carry out 
your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this 
e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS End of Disclaimer INFOSYS***

Re: Derby database

[Kristian Waagan]

On 05.05.12 12:13, Aristedes Maniatis wrote:

Running Derby 10.8, a customer has managed to corrupt their database
somehow (possibly by forcing the database to quit when it did not shut
down fast enough to their liking). I see that when I connect to it with
ij in embedded mode, I get the following:

ERROR 40XD2: Container Container(0, 36049) cannot be opened; it either
has been dropped or does not exist.


Hi Ari,

Do you have the stack trace for this exception?
The exception can be thrown from a few places, although the one in 
BaseDataFileFactory seems the most likely one.

Do you know when this exception was first seen?
Any other interesting errors in derby.log?
Are you aware of any power outages during this period, and if so, do you 
know if the write cache on the disk has been disabled? (it should be for 
durability)




My reading so far suggests that page 0 of some data with id 36049 is
corrupt. Is there further diagnostics I can do to determine whether this
data is just an index which could be rebuilt or something else? Are
there further tools I can use to progress? Can I excise the bad data and
manually replace it later?


The 0 refers to segment zero. Derby currently only supports one segment, 
so it is always zero.
The container 36049 would live in the file seg0/c8cd1.dat - do you have 
that file?


Your options for recovery depends on the data and the schema in the 
database, and would require full understanding of the relationships 
between the different tables etc:
  o if you have the file and it is corrupt, you, or the Derby 
developers, may be able to extract something useful out of it. Before 
share anything you have to determine if distributing the data is 
acceptable to your customer.
  o if the file is gone, you may be able to boot the database and 
investigate further, but the data would likely be inconsistent. The best 
thing that could happen here is that it is an index you can drop and 
recreate.




I can restore from backup, however the loss of a day's work since that
backup would be unfortunate. It does not appear that rollforward logs
were enabled.

mysql has a startup mode which allows it to start up in the face of data
corruption and then attempt recovery. Does Derby have a similar concept?


No, not in terms of missing and/or destroyed conglomerates.
Recovery in terms of redo/undo, for instance if the VM crashes, is 
performed automatically when the database is booted.



Regards,
--
Kristian



Regards

Ari

Re: Database shutdown not releasing file handles

[Kristian Waagan]

On 04.04.12 05:23, Trejkaz wrote:

We have the occasional user reporting that after shutting down the
database, they can see file handles still open.

We're using ;shutdown=true (or at least the DataSource equivalent of
it) to do this, and we were led to believe that this would be
sufficient to release all file handles.


Hi,

Yes, it should, but can you please clarify if you're talking about 
database shutdown or Derby system shutdown?

Also, do you know to what kinds of files Derby keeps open handles?



Is it possible that shutdown doesn't actually shut the database down?


Which version of Derby are you running with?


Regards,
--
Kristian



TX

Re: Database shutdown not releasing file handles

[Kristian Waagan]

On 04.04.12 09:27, Trejkaz wrote:

On Wed, Apr 4, 2012 at 4:46 PM, Kristian Waagan
kristian.waa...@oracle.com  wrote:

On 04.04.12 05:23, Trejkaz wrote:


We have the occasional user reporting that after shutting down the
database, they can see file handles still open.

We're using ;shutdown=true (or at least the DataSource equivalent of
it) to do this, and we were led to believe that this would be
sufficient to release all file handles.


Hi,

Yes, it should, but can you please clarify if you're talking about database
shutdown or Derby system shutdown?


Just individual database shutdown.


Also, do you know to what kinds of files Derby keeps open handles?


The files which are held are:

  seg0\c951.dat
  seg0\c861.dat
  seg0\c211.dat
  seg0\c71.dat
  seg0\c130.dat
  seg0\c731.dat
  seg0\cf0.dat

These do seem to be closed at some point *after* shutdown returns,
leading us to believe that shutdown doesn't necessarily shut down
now, but rather some time in the future.


Yes, if the handles are closed shortly after shutdown has returned this 
is unfortunately the currently expected behavior. Not sure if we have a 
JIRA for that, at least it has been discussed at some point.

Hopefully some time in the future is only a few seconds away.



The timing is like this:

  11:09:02.0444360 AM - close call presumably finishes around here
  11:10:00.8759376 AM - we attempt to rename the directory
  11:10:00.8760454 AM - first close call after our rename attempt
  11:10:12.4420715 AM - last close call after our rename attempt


Is it possible that shutdown doesn't actually shut the database down?


Which version of Derby are you running with?


v10.8.2.2.

Another probably-critical piece of information I missed was that the
OS is Windows. (If it had been Linux, we probably wouldn't ever notice
this sort of thing since we discovered the issue when trying to move a
file which was held open.)


Yes, that's typical, and the community sometimes observes the same thing 
when such a bug is introduced by someone developing on a non-Windows 
machine and then the nightlies fail on the Windows platform after the 
commit.



--
Kristian



Daniel

Re: Is it necessary to send shutdown in JDBC URL for Network Server clients?

[Kristian Waagan]

On 26.02.2012 17:34, Libor Jelinek wrote:

Hello everbody!
When running Derby Network Server scenario, is it neccessary for 
clients to create connection with shutdown attribute when 
application is about to terminate?


Or is enought to call java.sql.Connection.close() method? What is more 
gracefully?


Hi Libor,

Shutdown is an operation that affects all connections to the database, 
whereas Connection.close affects that connection only. If multiple 
clients are sharing a database, they would typically just close their 
connections.
If you know that the database won't be used any more / for a while, or 
the application about to terminate is the only client of the database, 
it is most graceful to shut down the database.


When a database is shut down Derby creates a database checkpoint and 
performs some other house-keeping tasks. If you don't shut down the 
database, Derby has to go through recovery on the next boot. Doing 
recovery is ok, but may cause the boot to take longer.


Note that shutting down a database doesn't shut down the network server, 
and further that stopping the network server doesn't trigger a clean 
shutdown of the databaes that are currently booted/active.



Regards,
--
Kristian



Thanks for all responses!
Libor

Re: Help derby not updating

[Kristian Waagan]

On 22.02.2012 22:27, Patricia Miriga wrote:


Hello,

I have been loading and running derby every morning at work. However, 
it has not been updating for a week now.


Please advise on how to do this.



Hi Patricia,

I have no idea what the community can help you with. Do you have any 
more information?



Regards,
--
Kristian


Thanks.

Patricia Miriga.

Re: Shutdown DB does not work / threadding issue

[Kristian Waagan]

On 18.02.2012 15:45, Karl Weber wrote:

Hi,

I am using Derby DB in an embedded environment. I am shutting down the DB and
the system using the following two methods (I have ommitted all try-catch-
finally stuff, logging etc.)

Shutting down the DB, where fDS is an EmbeddedDataSource40:

public void shutdownDB() {
fDS.setShutdownDatabase(shutdown);
fDS.getConnection().close();
}

Shutting down the system:

public void shutdownDerby() {
DriverManager.getConnection(jdbc:derby:;shutdown=true);
}

This works, if it is done on the thread that is shutting down my OSGi
component, which encapsulates Derby, i.e. the deactivate-method called by the
OSGi framework. This does, however, _not_ work, if it is put in a separate
thread in the deactivate-method, as in

new Thread(new Runnable() {
@Override
public void run() {
shutdownBD();
shutdownDerby();
}
}).start();

In this case, the Runnable stops at the statement

fDS.getConnection().close();


Hi,

When you say stops, do you mean the thread is blocking/hanging?
If so, can you obtain a stack trace?


Regards,
--
Kristian



No Throwable is thrown. Nevertheless, the DB does not seem to be stopped
correctly, as in the latter case (only) I find the two files dbex.lck and
db.lck in the DB folder.

Right before this last statement is called, I have around 12 threads
running, so the thread running the Runnable will most probably not be the last
thread in the JVM, if that matters.

For some reason I would like to shutdown things asynchroneously. What do I
miss here?

I am using Derby 10.8.2.2.

/Karl

Re: Unsubscribe

[Kristian Waagan]

On 03.02.2012 22:09, Gabriel Massip Fons wrote:
See http://db.apache.org/derby/derby_mail.html on how to unsubscribe.


Regards,
--
Kristian

Re: Selecting a few rows is really slow when ordering by primary key

[Kristian Waagan]

On 01.02.2012 14:51, Rick Hillegas wrote:

Hi Simon,

Glad to hear that regenerating the statistics fixed your problem. More 
inline...


On 2/1/12 4:07 AM, Simon Chatelain wrote:

...

But one small question remains, do you think that it is enough to run 
SYSCS_UPDATE_STATIS
TICS only once, or should I plan to run this procedure at regular 
interval (let's say once a week or once a month) in case the 
statistics are not correctly updated as new rows are inserted in the 
database ?


If you upgrade to 10.8, the statistics should be regenerated 
automatically. The statistics are supposed to be recalculated when the 
number of rows in the table is too far out of sync with the number of 
rows estimated by the statistics. Too far out of sync means that the 
ratio between the two numbers is more than e. If statistics are not 
being generated often enough, you can adjust the triggering ratio by 
setting the following Derby property:


derby.storage.indexStats.debug.lndiffThreshold

That property defaults to be 1.0. By setting it to a value between 0.0 
and 1.0, you should speed up the rate at which Derby recalculates 
statistics. By setting the property to a larger value, you should slow 
down the rate at which Derby recalculates statistics. Let us know if 
you think that statistics are not being regenerated often enough. This 
is a new feature which we are eager to tune and improve.


You may want to set derby.storage.indexStats.log=true, and more 
importantly, derby.storage.indexStats.debug.trace=true. Run you app for 
a while, then grep for {istat} in derby.log.

This will tell you if automatic calculation is being triggered.

Although strongly discouraged for production environments, you can try 
to set lndiffThreshold to zero, or a very small value, to schedule an 
update on every occasion, i.e. when you compile a statement like select 
... from tbl where indexed_column = ?.


Note that the istat daemon will only be triggered when the table changes 
in size. If you do updates that changes the index key distribution 
significantly you'll want to generate new statistics manually/periodically.



Regards,
--
Kristian



Hope this helps,
-Rick

Re: Selecting a few rows is really slow when ordering by primary key

[Kristian Waagan]

On 02.02.2012 11:14, Kristian Waagan wrote:
On 01.02.2012 14:51, Rick Hillegas wrote: 

 snip

On 2/1/12 4:07 AM, Simon Chatelain wrote:

 snip
You may want to set derby.storage.indexStats.log=true, and more 
importantly, derby.storage.indexStats.debug.trace=true. Run you app 
for a while, then grep for {istat} in derby.log.

This will tell you if automatic calculation is being triggered.

Although strongly discouraged for production environments, you can try 
to set lndiffThreshold to zero, or a very small value, to schedule an 
update on every occasion, i.e. when you compile a statement like 
select ... from tbl where indexed_column = ?.


If you want to test this manually, you will appreciate setting 
derby.storage.indexStats.debug.absdiffThreshold, and possibly 
derby.storage.indexStats.debug.createThreshold, to a lower value (both 
express number of rows, defaults are 1000 and 250 respectively).


Just for reference, testing stuff like this in ij can be done along 
these lines:
select * from sys.sysstatistics (note creationtimestamp / statistics, 
may want a join to get table name)

insert into ... (or delete from ...)
prepare q as 'select * from table where indexed_column = ?'
select * from sys.sysstatistics


--
Kristian



Note that the istat daemon will only be triggered when the table 
changes in size. If you do updates that changes the index key 
distribution significantly you'll want to generate new statistics 
manually/periodically.



Regards,

Re: DB2 9.7 Connection with Apache Derby - 10.5.3.1 on Windows 7 64-bit doesn’t work

[Kristian Waagan]

On 26.01.2012 11:07, Moni567 wrote:

Hello,

I used this
http://www.ibm.com/developerworks/data/library/techarticle/dm-0409kartha/
description  to setup the connection between DB2 and Apache Derby. This
Connection works with Apache Derby - 10.5.3.1 and DB2 9.5 on my other laptop
with Windows XP.
However, I tried to do the same on Windows 7 64-bit with DB2 9.7 and the
same Apache Derby version and it doesn’t work.

This is a copy of my derby.log content:

2012-01-25 19:01:33.014 GMT : Execution failed because of a Distributed
Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT arg  = 2135; Error Code Value =
e. Plaintext connection attempt from an SSL enabled client?
org.apache.derby.impl.drda.DRDAProtocolException: Execution failed because
of a Distributed Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT arg  = 2135;
Error Code Value = e. Plaintext connection attempt from an SSL enabled
client?
at org.apache.derby.impl.drda.DRDAConnThread.throwSyntaxrm(Unknown 
Source)
at org.apache.derby.impl.drda.DRDAConnThread.missingCodePoint(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.checkRequired(Unknown 
Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseACCRDB(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
2012-01-25 19:01:33.015 GMT Thread[DRDAConnThread_7,5,derby.daemons]
(DATABASE = TESTDB), (DRDAID = {10}), Execution failed because of a
Distributed Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT arg  = 2135; Error
Code Value = e. Plaintext connection attempt from an SSL enabled client?
2012-01-25 19:01:33.015 GMT : Execution failed because of a Distributed
Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT arg  = 2135; Error Code Value =
e. Plaintext connection attempt from an SSL enabled client?
org.apache.derby.impl.drda.DRDAProtocolException: Execution failed because
of a Distributed Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT arg  = 2135;
Error Code Value = e. Plaintext connection attempt from an SSL enabled
client?
at org.apache.derby.impl.drda.DRDAConnThread.throwSyntaxrm(Unknown 
Source)
at org.apache.derby.impl.drda.DRDAConnThread.missingCodePoint(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.checkRequired(Unknown 
Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseACCRDB(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.parseDRDAConnection(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)

Any help will be appreciated. :)


Hi Moni,

I'm afraid the short answer is that the DB2 drivers and Apache Derby 
aren't compatible any more (I don't remember when this broke, or when 
testing against DB2 drivers were abandoned).
Other who have more experience with DB2 drivers may be able to provide 
more input, but if you upgrade Derby the DB2 drivers are likely to stop 
working altogheter. The same may be true if you upgrade DB2 too.


For the community it may be helpful to know for what purpose you are 
using the DB2 drivers with Derby.



Regards,
--
Kristian



Thank you,
Moni

Re: List columns that make up an index

[Kristian Waagan]

On 20.01.12 03:49, ollie2308 wrote:


Hi

I am having trouble writing a query that uses the derby system tables to
return all  index names and the columns and their order that make up the
indexes.


Hi,

I'm not sure if we have a good solution for this.
You may find the information at [1] helpful, but to me it looks like 
you'll have to parse the value of the column DESCRIPTOR in 
SYS.SYSCONGLOMERATES to get what you want.



Regards,
--
Kristian

[1] http://wiki.apache.org/db-derby/ListKeyIndexes



Any help would be appreciated.

Thanks

Re: Shutting down in-memory Derby gracefully

[Kristian Waagan]

On 17.01.12 03:14, Pavel Bortnovskiy wrote:

Hello, all

can someone please also shed light on the following:

In addition to the problems described in the previous email (below), I also get 
intermittent  Database 'memory:test' not found errors while shutting down 
Derby:


Hi Pavel,

I don't know what's causing this intermittent error. The first thing 
that comes to mind is concurrent attempts to boot and/or drop the 
database. I don't know if this could be happening in you application?


Regarding the errors you are seeing when dropping the database, I think 
that's what you have to expect from Derby at this time when you have 
active connections at the time of the drop/shutdown. I don't think this 
is limited to the in-memory back end.
The only way to avoid these errors popping up is to coordinate the 
shutdown/drop from the application layer.


As a side note, I did some work on a mechanism that you may find useful. 
You can read the description of it under DERBY-4447 [1].



Regards,
--
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-4447



Error while shutting down:
java.sql.SQLException: Database 'memory:test' not found.
 at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
 at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
 at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown 
Source)   
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown 
Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.handleDBNotFound(Unknown 
Source)
 at org.apache.derby.impl.jdbc.EmbedConnection.init(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection30.init(Unknown Source)
 at org.apache.derby.impl.jdbc.EmbedConnection40.init(Unknown Source)
 at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
 at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
 at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
 at java.sql.DriverManager.getConnection(DriverManager.java:582)
 at java.sql.DriverManager.getConnection(DriverManager.java:207)
Caused by: java.sql.SQLException: Database 'memory:test' not found.
 at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
 at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
 Source)
 ... 21 more

This error doesn't happen all the time, it's intermittent and I suspect that it 
might have be somehow connected to other problems I've been having trying to 
shutdown in-memory Derby DB gracefully.

Can someone please help me understand what's causing this particular problem?

My code that's shutting down Derby is quite simple, and oddly it works most of the times, 
but sometimes it throws this  Database 'memory:test' not found  error!

Here is the excerpt of my shutdown code:

 private static final String DB_DROP_URL = 
jdbc:derby:memory:%DB_NAME%;drop=true;
 private static final String DB_SHUTDOWN_URL = jdbc:derby:;shutdown=true;

 private static class ShutdownAction {
 final private String m_url;
 final private int m_errorCode;
 final private String m_sqlState;

 private ShutdownAction(final String url, final int errorCode, final 
String sqlState) {
 m_url = url;
 m_errorCode = errorCode;
 m_sqlState = sqlState;
 }

 private void doAction(final String dbName) {
 final String url = m_url.replace(%DB_NAME%, dbName);
 logger.info(Executing Derby Shutdown Command:  + url);
 try {
 DriverManager.getConnection(url);
 } catch (SQLException e) {
 if ((e.getErrorCode() != m_errorCode) || 
(!m_sqlState.equals(e.getSQLState( {
 logger.error(Error while shutting down:, e);
 }
 }
 }
 }

 private static final ShutdownAction[] ACTIONS = {
 new ShutdownAction(DB_DROP_URL, 45000, 08006),
 new ShutdownAction(DB_SHUTDOWN_URL, 5, XJ015)
 };

 private String m_dbName;
 private String m_dbCreateURL;

 private NetworkServerControlPinger m_derbyServerControl;

 @Override
 final public void shutdown() {
 for (final ShutdownAction action : ACTIONS) {
 action.doAction(m_dbName);
 }

 m_derbyServerControl.shutdown();
 }



-Original Message-
From: Pavel Bortnovskiy [mailto:pbortnovs...@jefferies.com]
Sent: Friday, January 13, 2012 10:01 AM
To: Derby Discussion
Subject: RE: Shutting down in-memory Derby gracefully

Hello, Bryan:

Thank you for your prompt response. I will report it as per guidelines.

Would 

Re: [Unloading class org.apache.derby.exe.acf81e0010x0134x5183xd9d0x00001de8a8587]

[Kristian Waagan]

On 12/18/11 03:18 PM, Sameer Alwosaby wrote:


Dear

When My application  save records in java db ,  I received this

[Unloading class 
org.apache.derby.exe.acf81e0010x0134x5183xd9d0x1de8a8587]



[ snip - more unloading messages ]

[Unloading class 
org.apache.derby.exe.acf81e0010x0134x5183xd9d0x1de8a85856]


Please What is the problem ? and what is solution ?



Hi Sameer,

I don't think this is a problem in itself. The messages above look like 
JVM debug messages to me.


Derby compiles SQL queries into Java byte code, and this code is written 
to classes with names like above.
Note that you should be using prepared statements where possible, 
otherwise Derby may be generating more Java classes than necessary 
(hurting performance).



Hope this helps,
--
Kristian


Thanks

Re: SQLChar.getCollationKey NPE in index-stat-thread

[Kristian Waagan]

On 11.12.2011 19:42, Jean-Yves Linet wrote:

Hi,
I am upgrading Derby to last version 10.8.2.2 and I am stopped by what 
seams to be a bug.


After activation ot stats trace I get this :

Sun Dec 11 19:33:11 CET 2011 Thread[pool-3-thread-1,5,main] {istat} 
PROXIFLEX.IDAXX_RES: update scheduled, reason=[no stats, 
row-estimate=375] (queueSize=1)
Sun Dec 11 19:33:11 CET 2011 Thread[index-stat-thread,5,main] 
{istat,trace@26130360} worker thread started (xid=12049) 
[q/p/s=1/0/1,err:k/u/c=0/0/0,rej:f/d/o=0/0/0]
Sun Dec 11 19:33:11 CET 2011 Thread[index-stat-thread,5,main] 
{istat,trace@26130360} processing PROXIFLEX.IDAXX_RES
Sun Dec 11 19:33:11 CET 2011 Thread[index-stat-thread,5,main] {istat} 
runtime exception during normal operation

java.lang.NullPointerException
at org.apache.derby.iapi.types.SQLChar.getCollationKey(Unknown Source)
at 
org.apache.derby.iapi.types.WorkHorseForCollatorDatatypes.stringCompare(Unknown 
Source)
at 
org.apache.derby.iapi.types.CollatorSQLVarchar.stringCompare(Unknown 
Source)

at org.apache.derby.iapi.types.SQLChar.compare(Unknown Source)
at 
org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl$KeyComparator.compareWithPrevKey(Unknown 
Source)
at 
org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.updateIndexStatsMinion(Unknown 
Source)
at 
org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.generateStatistics(Unknown 
Source)
at 
org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.processingLoop(Unknown 
Source)
at 
org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.run(Unknown 
Source)

at java.lang.Thread.run(Thread.java:662)
Sun Dec 11 19:33:11 CET 2011 Thread[index-stat-thread,5,main] 
{istat,trace@26130360} worker thread exit 
[q/p/s=0/0/1,err:k/u/c=0/0/0,rej:f/d/o=0/0/0]


My JDBC connection url is 
: jdbc:derby:directory:db_name;territory=fr_FR;collation=TERRITORY_BASED:PRIMARY;create=true


If I remove territory and collation parameters I don't have the exception.

I didn't find bug report for this.
Should I make a JIRA entry or does a workaround exists ?


Hi Jean-Yves,

Yes, please create a JIRA issue to track this bug.



I also found undocumented property to disable the automatic statistics 
(derby.storage.indexStats.auto).
Can I set this property to false until the problem is solved without 
any side effect ?


Yes, you can disable the automatic statistics. This will basically make 
Derby 10.8 behave as older versions wrt index cardinality statistics. If 
you see performance drops on queries which involves tables that 
grow/shrink a lot, you should consider updating the statistics manually.
I believe work is in progress to document the istat feature properly. 
We're still looking for feedback from people on whether the feature is 
working satisfactory or not - in your case it obviously isn't due to 
what appears to be a bug related to collations.



Regards,
--
Kristian



Thanks

--
Jean-Yves LINET

Re: Move from embedded to network server

[Kristian Waagan]

On 12.09.11 11:21, johny_quest wrote:


Ok, lets see if I got you correctly:

I stop IBM Director.
I start the Derby Network Server giving it the database directory as home
(is this done within the start command?)
I then reconfigure IBM Director to connect with the network mode url (not
the embedded) and the new port number (because I set the Network server to
listen on port 1528).

I should then be able to connect simultaniously from more than one clients,
right?


Yes.

There is also another option;
Let IBM Director connect to Derby using the embedded driver and have it 
in addition start the network server. This can be done by setting a 
system property.


The advantages of the embedded server scenario above would be that the 
main application can connect using the embedded driver, and that the 
network server will live and die with the main application.
I don't know if this is something you want, but thought I'd mention the 
possibility.



Regards,
--
Kristian




José Ventura-3 wrote:


Derby only allows one instance *of the database software* (RDBMS) to open
the files in a directory at a time. According to the error you received,
there already is an instance of the RDBMS open on c:/Program
Files/IBM/Director/database (probably started by an embedded connection in
IBM Director).

Even if you use Derby Network Server, you will still be limited to one
instance *of the database software*, however this instance will accept
connections from any number of *clients*.

I think what you want to do is stop IBM Director, then start Derby Network
Server upon that directory, then configure IBM Director to use a client
connection instead of an embedded connection. That way, both Director and
other clients (such as razorSQL) will be able to connect, read and write
data to the database.

- José

On Fri, Sep 9, 2011 at 12:02 PM, johny_questivan_be...@yahoo.com  wrote:



Hello guys,
I have the following situation, that I have tried to solve for a couple
of
week but I can't seem to get it working.
Here is the deal.

I have a derby database that is running in embedded mode as a part of an
application (IBM Director software runs Apache Derby database by
default).

What I have to do, is to connect another type of software to that
database
using ODBC connector.

First I read about the fact, that only one connection is possibla at a
time.
I tried to connect to the database with razorSQL just to see if I can
connect, but the error is:

DERBY SQL error: SQLCODE:
-1, SQLSTATE: XJ040, SQLERRMC: Failed to start database 'c:/Program
Files/IBM/Director/database' with class loader
sun.misc.Launcher$AppClassLoader@11b86e7,
see the next exception
for details.::SQLSTATE: XSDB6Another instance of Derby may
have already booted the database C:\Program Files\IBM\Director\database.

After that I installed a new apache derby and set it in network mode. I
had
to change the port to 1528 because obviously the other instance is
listening
on 1527. Using the new setting I managed to connect to the server, create
database and tables etc. - it is working, but I still cannot connect to
the
needed database. The error is the same.

Please, can someone suggest a solution for this problem.
I will be very grateful.

--
View this message in context:
http://old.nabble.com/Move-from-embedded-to-network-server-tp32431926p32431926.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: Move from embedded to network server

[Kristian Waagan]

On 12.09.11 12:36, johny_quest wrote:


Hmm, this might be interesting. My goal is to change as little as possible in
IBM Director configuration, so this might be good for me. Otherwise there is
no difference because IBM Director is always ON, so the network server will
also be available at any time. Can you tell me how to achieve this
configuration.


I believe José Ventura already answered this, but here it is again:
http://db.apache.org/derby/docs/10.8/adminguide/radminconfigstartnetworkserver.html

BTW, all the docs can be found here: 
http://db.apache.org/derby/manuals/index.html



--
Kristian


Thank you.



Kristian Waagan-2 wrote:


On 12.09.11 11:21, johny_quest wrote:


Ok, lets see if I got you correctly:

I stop IBM Director.
I start the Derby Network Server giving it the database directory as home
(is this done within the start command?)
I then reconfigure IBM Director to connect with the network mode url (not
the embedded) and the new port number (because I set the Network server
to
listen on port 1528).

I should then be able to connect simultaniously from more than one
clients,
right?


Yes.

There is also another option;
Let IBM Director connect to Derby using the embedded driver and have it
in addition start the network server. This can be done by setting a
system property.

The advantages of the embedded server scenario above would be that the
main application can connect using the embedded driver, and that the
network server will live and die with the main application.
I don't know if this is something you want, but thought I'd mention the
possibility.


Regards,
--
Kristian




José Ventura-3 wrote:


Derby only allows one instance *of the database software* (RDBMS) to
open
the files in a directory at a time. According to the error you received,
there already is an instance of the RDBMS open on c:/Program
Files/IBM/Director/database (probably started by an embedded connection
in
IBM Director).

Even if you use Derby Network Server, you will still be limited to one
instance *of the database software*, however this instance will accept
connections from any number of *clients*.

I think what you want to do is stop IBM Director, then start Derby
Network
Server upon that directory, then configure IBM Director to use a client
connection instead of an embedded connection. That way, both Director
and
other clients (such as razorSQL) will be able to connect, read and write
data to the database.

- José

On Fri, Sep 9, 2011 at 12:02 PM, johny_questivan_be...@yahoo.com
wrote:



Hello guys,
I have the following situation, that I have tried to solve for a couple
of
week but I can't seem to get it working.
Here is the deal.

I have a derby database that is running in embedded mode as a part of
an
application (IBM Director software runs Apache Derby database by
default).

What I have to do, is to connect another type of software to that
database
using ODBC connector.

First I read about the fact, that only one connection is possibla at a
time.
I tried to connect to the database with razorSQL just to see if I can
connect, but the error is:

DERBY SQL error: SQLCODE:
-1, SQLSTATE: XJ040, SQLERRMC: Failed to start database 'c:/Program
Files/IBM/Director/database' with class loader
sun.misc.Launcher$AppClassLoader@11b86e7,
see the next exception
for details.::SQLSTATE: XSDB6Another instance of Derby may
have already booted the database C:\Program
Files\IBM\Director\database.

After that I installed a new apache derby and set it in network mode. I
had
to change the port to 1528 because obviously the other instance is
listening
on 1527. Using the new setting I managed to connect to the server,
create
database and tables etc. - it is working, but I still cannot connect to
the
needed database. The error is the same.

Please, can someone suggest a solution for this problem.
I will be very grateful.

--
View this message in context:
http://old.nabble.com/Move-from-embedded-to-network-server-tp32431926p32431926.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: Derby client fails to connect to server every time after updating to 10.8.1.2

[Kristian Waagan]

On 02.09.11 07:40, Trejkaz wrote:

Hi all.

We have tests (and apps, naturally) which start up the network server
and then test that connecting to it works.  One test runs our wrapping
around the server and uses bare JDBC code to verify that it's
connectable.  The other test runs our wrapping around the client and
uses bare NetworkServerControl calls to start the server.  Both of
these tests fail after updating from 10.6.1.0 -  10.8.1.2.


Hi,

The code below works for me with 10.8.1.2. I also added code to select 
from a system table - just to get some output.

I don't have to set the user either (I think a default will be used).

So why is it working for me, but not for you:
 o which Derby jars are you running off?
 o what are typical values of the scratch variable?
 o which platform are you running on?
 o which JVM is being used?
 o anything else in the test framework (or apps) that may cause this?
   (i.e., is configuration settings being picked up from somewhere?)


Regards,
--
Kristian



The error I am getting:

Fri Sep 02 15:21:40 EST 2011 : Execution failed because of a
Distributed Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT arg  = 115a;
Error Code Value = 9. Plaintext connection attempt from an SSL enabled
client?
org.apache.derby.impl.drda.DRDAProtocolException: Execution failed
because of a Distributed Protocol Error:  DRDA_Proto_SYNTAXRM; CODPNT
arg  = 115a; Error Code Value = 9. Plaintext connection attempt from
an SSL enabled client?
at 
org.apache.derby.impl.drda.DRDAConnThread.throwSyntaxrm(DRDAConnThread.java:517)
at 
org.apache.derby.impl.drda.DRDAConnThread.tooBig(DRDAConnThread.java:8160)
at 
org.apache.derby.impl.drda.DRDAConnThread.parseEXCSAT(DRDAConnThread.java:1583)
at 
org.apache.derby.impl.drda.DRDAConnThread.exchangeServerAttributes(DRDAConnThread.java:1148)
at 
org.apache.derby.impl.drda.DRDAConnThread.sessionInitialState(DRDAConnThread.java:672)
at 
org.apache.derby.impl.drda.DRDAConnThread.run(DRDAConnThread.java:283)

I joined the unit tests to form one test which doesn't use any of our
own code, and the same error occurs.  The test does require a
'scratch' field to be set up... which is unfortunately still part of
our test framework. :/

 @Test
 public void test() throws Exception
 {
 File dbDir = new File(scratch, db);

 EmbeddedConnectionPoolDataSource ds1 = new
EmbeddedConnectionPoolDataSource();
 ds1.setDatabaseName(dbDir.getAbsolutePath());
 ds1.setCreateDatabase(create);
 ds1.getConnection().close();

 NetworkServerControl server = new
NetworkServerControl(InetAddress.getByName(127.0.0.1), 27000);
 try
 {
 server.start(null);

 ClientConnectionPoolDataSource ds = new
ClientConnectionPoolDataSource();
 ds.setServerName(127.0.0.1);
 ds.setPortNumber(27000);
 ds.setDatabaseName(dbDir.getAbsolutePath());
 ds.setUser(bob);

 Connection connection = ds.getConnection();
 connection.close();
 }
 finally
 {
 server.shutdown();

 // clean up after ourselves
 EmbeddedConnectionPoolDataSource ds2 = new
EmbeddedConnectionPoolDataSource();
 ds2.setDatabaseName(dbDir.getAbsolutePath());
 ds2.setShutdownDatabase(shutdown);
 try { ds2.getConnection(); } catch (SQLException e) { /*
expected */ }
 }
 }

The same code still works fine on 10.6.1.0.

Additionally, on 10.6.1.0 I don't have to set the user.  I searched
the changelog and couldn't see anything about a user now being
mandatory, but I assume this has changed for security reasons?

TX

Re: Derby (in-memory) and Hibernate

[Kristian Waagan]

On 31.08.11 04:08, Pavel Bortnovskiy wrote:

Are there any issues in using Derby (in-memory only mode) with Hibernate?
Is it fully supported?


Hi Pavel,

I'm not aware of any, but I'm not using Hibernate a lot. Maybe someone 
else has experience with this, or perhaps you could ask on a Hibernate 
forum?


As for the in-memory functionality in Derby, basically the only thing 
that differs from the on-disk functionality is the low level store - 
data is kept in memory instead of being written to disk. Very few 
changes were made to the layers above (but handling of the URL 
subsubprotocol had to be implemented, for instance).



Regards,
--
Kristian



Thank you.
Pavel.



Jefferies archives and monitors outgoing and incoming e-mail. The
contents of this email, including any attachments, are confidential to
the ordinary user of the email address to which it was addressed. If you
are not the addressee of this email you may not copy, forward, disclose
or otherwise use it or any part of it in any form whatsoever. This email
may be produced at the request of regulators or in connection with civil
litigation. Jefferies accepts no liability for any errors or omissions
arising as a result of transmission. Use by other than intended
recipients is prohibited. In the United Kingdom, Jefferies operates as
Jefferies International Limited; registered in England: no. 1978621;
registered office: Vintners Place, 68 Upper Thames Street, London EC4V
3BJ. Jefferies International Limited is authorised and regulated by the
Financial Services Authority.

Re: Bug report for Apache Derby 10.4.x and onwards

[Kristian Waagan]

On 12.08.11 11:41, srgrocks wrote:

Hello Apache Derby team ,

This is to bring to your kind attention the fact that Apache Derby
10.6.2.1 , the version I am using is giving a error on using the ON
UPDATE CASCADE statement .


Hello,

ON UPDATE CASCADE is not supported for foreign keys.
From the Reference Manual under CREATE TABLE statement:

Foreign keys

If the generated column is part of a foreign key that references another 
table, the referential action must not specify SET NULL or SET DEFAULT, 
and the update rule must not specify ON UPDATE CASCADE.



I'm not sure if the documentation is accurate - I don't think ON UPDATE 
CASCADE works even if no generated column is used.


To increase the chances of getting this feature implemented, you can 
pick up or vote on DERBY-735 [1].



Regards,
--
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-735



Here is the SQL statement in Java DB

create table book
(
Booking_ID varchar(255) primary key,
Customer_ID varchar(255) references customer *ON UPDATE CASCADE* ON
DELETE CASCADE,
Hotel_ID varchar(255) references hotel ON UPDATE CASCADE ON DELETE
CASCADE,
Date_Of_Booking varchar(255),
Check_In_Date varchar(255),
Check_Out_Date varchar(255),
No_Of_Rooms_Booked int ,
Total_Price varchar(255),
Status varchar(255)
)

The bug thrown is :

Error code -1, SQL state 42X01: Syntax error: Encountered CASCADE at
line 4, column 58.
Line 1, column 1





 snip 

Re: Table exists in same JVM after Derby is shutdown

[Kristian Waagan]

On 09.08.11 17:14, Pavel Bortnovskiy wrote:

Thank you, Kristian.

Can you please clarify some of the points you made (or point me to an
online primer that describes it):
- what is the difference between shutting down a specific db and the
whole engine?


Shutting down a specific database will close all open handles to that 
database and free up resources associated with the database. The next 
connection attempt to this database will cause it to be booted again.
When a database isn't booted, i.e. it has been shut down, it can also 
be safely copied using OS utilities [1].


Shutting down the Derby engine usually makes the Derby classes eligible 
[2] for garbage collection. You won't be able to obtain a new connection 
to a Derby database before you reload the engine/driver [3].

Shutting down the engine also shuts down all booted databases.


- what does it mean that shutting down the Derby engine which runs an
in-memory-only DB doesn't drop tables?


It means that the in-memory database will be kept in memory, just as an 
on-disk database will be kept on disk.
The in-memory databases are stored in a static variable. The ways to get 
rid of them are to explicitly drop them, to have the Derby classes 
garbage collected [2] , or to exit the JVM.



Are you saying that if that engine were to be restarted, the tables
would be accessible again?


Yes, all in-memory databases would be accessible again.


Does it also mean that the memory taken by the engine and its tables
doesn't get reclaimed until it's shutdown (or dropped)?


Shutting down the engine will free up resources like the page cache, and 
most, but not all, instances of Derby classes will be garbage collected.
If in-memory databases have been created, but not dropped, these will 
remain in memory.



Regards,
--
Kristian

[1] There are other ways to safely copy a database, for instance 
freezing it while booted or using the Derby backup routines.


[2] To actually unload the Derby classes, the class loader that loaded 
them has to be eligible for gc too. Simply shutting down the engine is 
not sufficient (see also [3]). Note the difference between garbage 
collecting instances of Derby classes and unloading the Derby classes 
themselves.


[3] This changed somewhat with the addition of the 
deregister=*true*|false attribute. If false, the driver will be left 
registered even if the engine is shut down.




Regards,
Pavel.


From:   Kristian Waagan kristian.waa...@oracle.com
To: derby-user@db.apache.org
Date:   08/09/2011 01:42 AM
Subject:Re: Table exists in same JVM after Derby is shutdown






On 08.08.11 16:41, Pavel Bortnovskiy wrote:
  Thank you, Dag.
 
  It looks like in order to get this to work, I had to make two calls to
  DriverManager.getConnection.
  Combining /shutdown=true;drop=true/ didn't work.
  So, is this the correct implementation (if in-line image doesn't work,
  please see attachment):

Hello Pavel,

To simply drop an in-memory database, you only need the drop attribute.
Shutdown of the specified database happens as part of the drop operation.

It is correct that you need two calls to DriverManager.getConnection if
you want to drop a database and additionally shut down the Derby engine.
For clarity, note that there is a difference between shutting down a
specific database and the Derby engine/system.

Further, shutting down an in-memory database does not drop it, neither
does shutting down the Derby engine. I don't know, maybe it would make
sense to drop all in-memory databases when the Derby system is shut down
- but that's not being done currently.


Hope this helps,
--
Kristian

 
 
 
 
 
  From: dag.wan...@oracle.com (Dag H. Wanvik)
  To: Derby Discussion derby-user@db.apache.org
  Date: 08/05/2011 06:36 PM
  Subject: Re: Table exists in same JVM after Derby is shutdown
 
 
  
 
 
 
  Pavel Bortnovskiy pbortnovs...@jefferies.com writes:
 
   1) I create an in-memory database and then a table in it.
   Then the database is shut down.
   I would expect that the shutdown effectively cleans everything up.
   Is it not so?
 
  Ah, I misunderstood you Pavel. Have a look here:
 
  http://wiki.apache.org/db-derby/InMemoryBackEndPrimer
 
  As you can see, you need to provide the drop=true connection
  attribute to wipe the data.
 
  Thanks,
  Dag
 
  
   Thanks,
   Pavel.
  
  
  
   From:
   dag.wan...@oracle.com (Dag H. Wanvik)
   To:
   Derby Discussion derby-user@db.apache.org
   Date:
   08/05/2011 05:57 PM
   Subject:
   Re: Table exists in same JVM after Derby is shutdown
  
  
  
   Pavel Bortnovskiy pbortnovs...@jefferies.com writes:
  
   Hello, all:
  
   While executing a bunch of JUnit tests within the same JVM (all
  executed
  
   by IntelliJ IDEA) I started seeing strange and unexpected errors
   occurring
   .
   Upon closer inspection, I noticed that in many

Re: Table exists in same JVM after Derby is shutdown

[Kristian Waagan]

On 08.08.11 16:41, Pavel Bortnovskiy wrote:

Thank you, Dag.

It looks like in order to get this to work, I had to make two calls to
DriverManager.getConnection.
Combining /shutdown=true;drop=true/ didn't work.
So, is this the correct implementation (if in-line image doesn't work,
please see attachment):


Hello Pavel,

To simply drop an in-memory database, you only need the drop attribute. 
Shutdown of the specified database happens as part of the drop operation.


It is correct that you need two calls to DriverManager.getConnection if 
you want to drop a database and additionally shut down the Derby engine. 
For clarity, note that there is a difference between shutting down a 
specific database and the Derby engine/system.


Further, shutting down an in-memory database does not drop it, neither 
does shutting down the Derby engine. I don't know, maybe it would make 
sense to drop all in-memory databases when the Derby system is shut down 
- but that's not being done currently.



Hope this helps,
--
Kristian







From:   dag.wan...@oracle.com (Dag H. Wanvik)
To: Derby Discussion derby-user@db.apache.org
Date:   08/05/2011 06:36 PM
Subject:Re: Table exists in same JVM after Derby is shutdown






Pavel Bortnovskiy pbortnovs...@jefferies.com writes:

  1) I create an in-memory database and then a table in it.
  Then the database is shut down.
  I would expect that the shutdown effectively cleans everything up.
  Is it not so?

Ah, I misunderstood you Pavel. Have a look here:

http://wiki.apache.org/db-derby/InMemoryBackEndPrimer

As you can see, you need to provide the drop=true connection
attribute to wipe the data.

Thanks,
Dag

 
  Thanks,
  Pavel.
 
 
 
  From:
  dag.wan...@oracle.com (Dag H. Wanvik)
  To:
  Derby Discussion derby-user@db.apache.org
  Date:
  08/05/2011 05:57 PM
  Subject:
  Re: Table exists in same JVM after Derby is shutdown
 
 
 
  Pavel Bortnovskiy pbortnovs...@jefferies.com writes:
 
  Hello, all:
 
  While executing a bunch of JUnit tests within the same JVM (all
executed
 
  by IntelliJ IDEA) I started seeing strange and unexpected errors
  occurring
  .
  Upon closer inspection, I noticed that in many of those tests tables
  with
  the same names are attempted to be created.
  Then I realized that although Derby is shutdown and then re-created,
the
 
  tables remain, thus causing collisions.
 
  The create=true connection attribute is ignored (with a warning) if
  the database with the same name alrady exists.
 
 
  I've created a digest (attached) which is executed as one JUnit test to
  illustrate what I'm seeing.
  The behavior I would expect is that once Derby is shutdown, no tables
  would remain in the JVM, and if a database (with the same name) is
  re-created, it would be a tabula rasa.
 
  The tables are no longer in memory (or should not be unless you found a
  bug), but they are not erased from the disk image of the database. As
  per the above, one would need to explicitly delete it using OS file
  system tools for the data to be cleared. Some JUnit tests delete tables
  in TestCase#tearDown, others use singleUseDatabaseDecorator to isolate
  itself from the rest of the tests. The tests not necessarily very
  consistent in their patterns for this..
 
  Dag
 
 
  Can you please let me know whether my expectations are erroneous and
  whether I should find workarounds (albeit trivial to implement).
  However, what would concern me in that case is that the tables and the
  data remain in the JVM, thus consuming memory
  (and, if unused, creating memory leaks).
 
  Thank you,
 
  Pavel.
 


 snip 



Jefferies archives and monitors outgoing and incoming e-mail. The
contents of this email, including any attachments, are confidential to
the ordinary user of the email address to which it was addressed. If you
are not the addressee of this email you may not copy, forward, disclose
or otherwise use it or any part of it in any form whatsoever. This email
may be produced at the request of regulators or in connection with civil
litigation. Jefferies accepts no liability for any errors or omissions
arising as a result of transmission. Use by other than intended
recipients is prohibited. In the United Kingdom, Jefferies operates as
Jefferies International Limited; registered in England: no. 1978621;
registered office: Vintners Place, 68 Upper Thames Street, London EC4V
3BJ. Jefferies International Limited is authorised and regulated by the
Financial Services Authority.

Re: Mistake CLASSAPATH

[Kristian Waagan]

On 28.06.11 16:40, Wolfram Miksch wrote:

Syntax for the derbyrun.jar file.
http://db.apache.org/derby/docs/dev/getstart/rgsderbyrunjarsyntax.html#rgsderbyrunjarsyntax

Found problem in above file: you wrote CLASSAPATH instead of CLASSPATH


Vielen Dank, Wolfram.

I have fixed the typo (revision 1140796).
It will take a little while before the change makes it into the 
published online docs.



Cheers,
--
Kristian



lg, Wolfram Miksch

Re: SYSCS_UTIL.SYSCS_BACKUP_DATABASE failing-urgent

[Kristian Waagan]

On 09.06.11 09:27, Brett Wooldridge wrote:

Sorry I wasn't clearer.  A thead dump is a snapshot of all running
threads in the system, not a stacktrace of failures.

You can generate a threaddump on Windows by running your server from
the command line, and when the hang occurs during backup, pressing
CTRL-Break (CTRL key and Break key at the same time).  This will
produce a threaddump in your console (cmd.exe) window.  Copy/paste the
threaddump somewhere we can take a look at it.

If you are on Linux, you can produce a threaddump by sending the
process a QUIT signal:

kill -QUIT process_id


You may also be able to use jstack.

On another matter, are there any relevant error messages in derby.log - 
the Derby log on the server side?



Regards,
--
Kristian



Regards,
Brett


On Thu, Jun 9, 2011 at 4:01 PM, dvijendervijender.devak...@db.com  wrote:


Hi Mike, Brett,

Attached thread dump. Here the database size is 20 GB.


Brett Wooldridge-2 wrote:


Do you have a thread dump during the hang?  What
OS are you running on?

Brett

Sent from my iPhone

On Jun 7, 2011, at 16:38, Vijender Devakarivijender.devak...@db.com
wrote:

Hi,

Can you respond to this as this is very urgent.

Best Regards,

Vijender D,
Deutsche Bank
Ph: +65 6423 8530
Mobile: +65 94517994


This mail is transmitted to you on behalf of [HCL].
Diese Post wird Ihnen im Namen der [HCL] ubermittelt
*

graycol.gifVijender Devakari/ext/dbcom



 *Vijender Devakari/ext/dbcom*

 06/03/2011 04:28 PM

  ecblank.gif
To
ecblank.gif
derby-user@db.apache.orgecblank.gif
cc
ecblank.gif
  ecblank.gif
Subject
ecblank.gif
SYSCS_UTIL.SYSCS_BACKUP_DATABASE failingecblank.gifecblank.gifHi
Team,

we are using below proc for backing up the database, but this call is
hanging when the Database is size more like 3GB, 4GB, ...

Can you let me know in which case we can use this and also what else we
need
to use to backup database incase if the siz is more.

CALL SYSCS_UTIL.SYSCS_BACKUP_DATABASE(?)

Best Regards,

Vijender D,
Deutsche Bank
Ph: +65 6423 8530
Mobile: +65 94517994


This mail is transmitted to you on behalf of [HCL].
Diese Post wird Ihnen im Namen der [HCL] ubermittelt
*


---

This e-mail may contain confidential and/or privileged information. If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail. Any
unauthorized copying, disclosure or distribution of the material in this
e-mail is strictly forbidden.



http://old.nabble.com/file/p31806974/Logs.zip Logs.zip
--
View this message in context: 
http://old.nabble.com/Re%3A-SYSCS_UTIL.SYSCS_BACKUP_DATABASE-failing-urgent-tp31789732p31806974.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.

Re: Feedback derby_downloads.html

[Kristian Waagan]

On 06.06.11 11:36, Richter, Tania wrote:

Hello,

we are using Derby 10.8.1.2 in our project. Now we have problems that
are already reported in issue Derby-5098. As we can see this issue is
fixed, but there is no official release right now (10.8.1.3 is supposed
to come out mid September). Is there a possibility to get a pre-release
or is the only possibility to get the source code and create it on our own?


Hi Tania,

You can always try to convince one of the committers to drive the 
release process for you, but since 10.8.2 has already been planned I 
suspect you'll get your hands on the fix quicker by building head of 
10.8 yourself.
When I checked earlier today, there had been 18 commits on the 10.8 
branch since 10.8.1 was released. You could also start with the 10.8.1.2 
code base and apply only the fix you need to minimize risk.


Depending on your needs and requirements, one of these might help:
http://wiki.apache.org/db-derby/SnapshotsAndQuickBuilds (see Making a 
(private) Quick Build of Release Artifacts)

http://wiki.apache.org/db-derby/DerbySnapshotOrRelease

Of course, the risk of problems is higher when you build your own 
release, since the bits haven't been tested and vetted by the community.


Maybe there are other users on the list that can share their experiences 
with rolling their own builds?



Regards,
--
Kristian

[1] svn log -r 1095077:HEAD 
https://svn.apache.org/repos/asf/db/derby/code/branches/10.8





Regards

Tania Richter
Wincor Nixdorf International GmbH
BD SWD 5
Gutenbergstr.12
63263 Neu Isenburg, Germany
fon : +49-6102-202-1562
fax : +49-6102-202-1699
mail :mailto:tania.rich...@wincor-nixdorf.com
net : http://www.wincor-nixdorf.com/
WINCOR NIXDORF International GmbH
Sitz der Gesellschaft: Paderborn
Registergericht Paderborn HRB 3507
Geschäftsführer: Eckard Heidloff (Vorsitzender), Stefan Auerbach, Dr.
Jürgen Wunram
Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller
Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 - WEEE-Reg.-Nr.
DE44477193

*WINCOR NIXDORF International GmbH*
Sitz der Gesellschaft: Paderborn
Registergericht Paderborn HRB 3507
Geschäftsführer: Eckard Heidloff (Vorsitzender), Stefan Auerbach,
Dr. Jürgen Wunram
Vorsitzender des Aufsichtsrats: Karl-Heinz Stiller
Steuernummer: 339/5884/0020 - Ust-ID Nr.: DE812927716 -
WEEE-Reg.-Nr. DE44477193

Diese E-Mail enthält vertrauliche Informationen.
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail
irrtümlich erhalten haben,
informieren Sie bitte sofort den Absender und vernichten Sie diese
E-Mail.
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail
ist nicht gestattet.

This e-mail may contain confidential information.
If you are not the intended recipient (or have received this e-mail
in error)
please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Re: Derby network server

[Kristian Waagan]

On 02.05.11 15:13, John English wrote:

I have a webapp which runs a networked Derby server. I want to be able
to access it from a separate app. The webapp starts the server from the
contextInitialized() method of a ServletContextListener:

  server = new NetworkServerControl();
  server.start(null);

which works fine. However, when I try to open a second connection with
IJ it gives me this response:

  ij connect '/foo/db;dataEncryption=true;bootPassword=xyz';
  ERROR XJ040: Failed to start database '/foo/db', see the next
  exception for details.
  ERROR XSDB6: Another instance of Derby may have already booted the
  database C:foo\db.
  ij

Is there something I'm missing here?


Hi John,

Looks to me like you are trying to access, or more specifically boot, 
the database with the embedded driver. This won't work since the network 
server has already booted the database [1].
The only change required should be to add the host name to the 
connection URL (localhost?), i.e. something like 
jdbc:derby://localhost[:port]/db[;attributes]. You will then get two 
(or more) connections to the database, all using the client driver and 
going through the network server.



Regards,
--
Kristian

[1] I'm assuming a connection to the database has already been made 
through the network server. If you connect with the embedded driver 
first, the connection through the network server will fail instead.




TIA,


 John English | My old University of Brighton home page is still here:
  | http://www.it.brighton.ac.uk/staff/je

Re: BLOB streaming

[Kristian Waagan]

On 18.02.2011 14:59, Brett Wooldridge wrote:

The question is, is it still fully materialized on the server before
streaming to the client?


No, it's not.
Ideally, the only time a BLOB would be fully materialized is if 
ResultSet/Blob.getBytes() is called.
There used to be some exceptions (triggers and a few special queries?), 
but I believe at least most of them have been addressed by now.


Other considerations come into play when you are updating BLOBs, but in 
that case large BLOBs are stored intermediately on disk if required.



Regards,
--
Kristian


Brett

Sent from my iPhone

On Feb 18, 2011, at 22:11, Knut Anders Hatlenknut.hat...@oracle.com  wrote:


Brett Wooldridgebrett.wooldri...@gmail.com  writes:


Hi all,


I just came across this in the manual, and if accurate raises some
concerns for me:

For applications using the client driver, if the stream is stored in a
column of a type other than LONG VARCHAR or LONG VARCHAR FOR BIT DATA,
the entire stream must be able to fit into memory at one time. Streams
stored in LONG VARCHAR and LONG VARCHAR FOR BIT DATA columns do not
have this limitation.

This seems to imply that if I have a BLOB containing 1GB of data, and
I'm using the client driver, the result cannot be streamed?
Can this possibly be correct? �Given the apparent limit of VARCHAR of
~32K, is there no way to stream large data to a client?

A lot of work went into Derby 10.2 and Derby 10.3 to avoid the need to
materialize LOBs on the client, so I believe this statement isn't true
anymore. I've filed https://issues.apache.org/jira/browse/DERBY-5056 to
update the manual.

Thanks,

--
Knut Anders

Re: commands describe and show failed in eclipse derby plugin SQL editor

[Kristian Waagan]

On 04.01.11 07:05, Bryan Pendleton wrote:

On 01/03/2011 09:56 PM, Bruce Wen wrote:
I used derby eclipse plugin from 
http://db.apache.org/derby/releases/release-10.7.1.1.cgi.


I think that the best course of action here would be for you to file a 
new
issue in Jira, including the information that you've provided in this 
email

thread so far, and hopefully somebody one of the Derby developers who is
familiar with the Eclipse plugin can help resolve this.


For clarity, can you confirm that the problem isn't that you are trying 
to execute ij-specific commands through the Derby JDBC-driver(s)?



Regards,
--
Kristian



http://db.apache.org/derby/DerbyBugGuidelines.html

thanks,

bryan

Re: Error XSDBB When Trying to Access DB From Windows, But Works In UNIX

[Kristian Waagan]

 On 10.12.10 16:35, BEK1976 wrote:

Hi,



Hi,

What I'm asking  may very well be a red herring, but it would be nice to 
get it confirmed anyway.
What's the underlying file system on the Windows machine, and what's the 
maximum allowed file size for that file system?



Regards,
--
Kristian


I'm trying to debug a strange error that just started occurring recently.
This past weekend, our Derby database began reporting error XSLA7 (Cannot
redo operation null in the log) and XSDBB (Unknown Page Format) when we try
to access data from the DB using a MATLAB GUI front-end tool that calls into
our internally-developed DB table utilities.  When we run our actual
application, which runs on IBM AIX using the same database table utilities,
the database works fine.

Note that these MATLAB developed tools have been working fine for quite some
time, and the errors seem to correlate to database size.  When we add enough
data to this one particular table to push one of the internal derby files
over 2 GB, that's when the error starts occurring.  It's also worth noting
is that we are running AIX on 64-bit machines, and we are running Windows on
32 bit machines.  Is there a known compatibility issue with Derby with
respect to populating a database in a 64 bit environment and then accessing
it in a 32 bit environment?

As another test, I tried connecting to the database via a portion of our
application code in Eclipse, bypassing the MATLAB tools.  We again get the
error when trying to query the database.

This problem first began occurring under Derby 10.5.1.  However, after
seeing this Derby error report
(https://issues.apache.org/jira/browse/DERBY-4239), which exactly matched
our errors, I upgraded to Derby 10.5.3 and reverted back to a version of the
database that did not exhibit the problem.  Unfortunately, that did not seem
to help -- once the DB got larger once again, we got the mostly the same
errors.  (I no longer get XSLA7, but still get XSDBB with the all-zeros hex
dump.)

Any help would be most appreciated!
Thanks,
-Brian

Re: Derby : No suitable driver found for jdbc:derby:db;create=true

[Kristian Waagan]

 On 02.12.10 16:35, BeNdErR wrote:

I modifyed as you told me, that's the error print I get in the client window:

SQLState:08001
Error Code:0
Message:No suitable driver found for jdbc:derby:db;create=true


The rmid window, for the activatable server, shows this error:


Maybe you already said, but what kind of server are you using?

I have experienced that if you include a jar-file ([1]) on the classpath 
from two different locations, things will go crazy. This was using an 
appserver, where the jar was loaded both from a system-wide lib 
directory and a domain-specific lib directory.



--
Kristian

[1] Not sure if this is a general problem, or a Derby-specific problem.



Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:out: DBMS.java
ClassNotFoundException Error: org.apache.derby.jdbc.EmbeddedDriver
Thu Dec 02 16:22:34 CET
2010:ExecGroup-0:err:java.lang.ClassNotFoundException:
org.apache.derby.jdbc.EmbeddedDriver
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.net.URLClassLoader$1.run(URLClassLoader.java:202)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.security.AccessController.doPrivileged(Native Method)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.net.URLClassLoader.findClass(URLClassLoader.java:190)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.lang.ClassLoader.loadClass(ClassLoader.java:307)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.lang.Class.forName0(Native Method)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.lang.Class.forName(Class.java:169)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
tmp.DBMS.init(DBMS.java:24)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:out:Apro la connessione con il DB.
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
tmp.AuthServer.register(AuthServer.java:79)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.lang.reflect.Method.invoke(Method.java:597)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.rmi.transport.Transport$1.run(Transport.java:159)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.security.AccessController.doPrivileged(Native Method)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.rmi.transport.Transport.serviceCall(Transport.java:155)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
Thu Dec 02 16:22:34 CET 2010:ExecGroup-0:err:   at
java.lang.Thread.run(Thread.java:662)

the first error is thrown when he enters the DBMS constructor. In the setup
code of the activatable server I used this setting to tell him where he can
find *.class he needs:
prop.put(java.class.path, /home/user/app/lib/);
because rmid does not accept a -classpath option, I assume that putting the
classpath into the code is an alternative method..
I never mentioned before this error because, sincerely and stupidly, I never
watched the rmid window, focusing only on the client error.

I have a question: does the fact that I run every piece of the app (server,
rmid, client) in different shells? I mean, running (or trying) derby in one
shell, lets the others interact with it?
Another question: my scope is to build this app with a database, then move
ALL the codes into another pc, without installing derby (I can't because on
that pc I'm not the admin). I thought was enough to move all the derby
libraries in a folder, and then run everywhere I need my application.. Am I
all abroad?

Hoping I'm not wasting your time in a useless thing, thanks a lot

Re: Derby : No suitable driver found for jdbc:derby:db;create=true

[Kristian Waagan]

On01.12.2010 19:45, BeNdErR wrote:

@bryan
I tryed to run ij instead of tmp.Setup, as you said, then I tryed to connect
to a new db, that's what I got:

ij  CONNECT 'ciao;create=true';
ERRORE 08001: No suitable driver found for ciao;create=true


Note that unless you have configured Derby to use a specific protocol 
(impossible to say based on the output above), the connection URL has to 
be 'jdbc:derby:ciao;create=true'.
ij is a rather general tool, and you could use it to connect to another 
database, in which case 'derby' would have to be replaced and maybe that 
driver would need different connection attributed.


Another example is if you want to use the in-memory back end with Derby, 
in which case the URL would be something like 
'jdbc:derby:memory;ciao;create=true';



Regards,
--
Kristian


ij


so something is still wrong :\

running org.apache.derby.tools.sysinfo I have:


-- Informazioni su Java --
Versione Java:   1.6.0_22
Fornitore Java:  Sun Microsystems Inc.
Home Java:   /usr/lib/jvm/java-6-sun-1.6.0.22/jre
Percorso classi Java:
/home/user/app/lib/derby.jar:.:/home/user/app/lib/derbytools.jar:.
Nome SO: Linux
Architettura SO: i386
Versione SO: 2.6.35-23-generic
Nome utente Java: user
Home utente Java: /home/user
Dir utente Java: /home/user/app/server
java.specification.name: Java Platform API Specification
java.specification.version: 1.6
java.runtime.version: 1.6.0_22-b04
- Informazioni su Derby 
JRE - JDBC: Java SE 6 - JDBC 4.0
[/home/user/app/lib/derby.jar] 10.6.2.1 - (999685)
[/home/user/app/lib/derbytools.jar] 10.6.2.1 - (999685)
--
- Informazioni sulla locale -
Locale attuale:  [italiano/Italia [it_IT]]
E' stato trovato il supporto per la locale: [it]
 versione: 10.6.2.1 - (999685)
--

If you need something to be translated, tell me.
In ~/home/user/app/lib/ folder I have these files: derby.jar, derby.war.
derbyclient.jar, derbyLocale_it.jar, derbynet.jar, derbyrun.jar,
derbytools.jar.
In the original derby installation folder, in /lib/ there are more file than
I have in my app/lib, because I removed all the *.jar for the other
languages (to be nerdy, i removed all derbyLocale_xy.jar where (xy != it). I
don't think the problem it's there anyway...

To anyone who is still reading and replying to this thread, thanks a lot,
you're very kind.

Re: ERROR 08004: Connection refused : Invalid authentication.

[Kristian Waagan]

 On 29.11.10 09:32, Sonny Laskar wrote:

Dear Thomas,
Thanks for your inputs.
I am not much familiar with the APIs.

Let me tell you how I am proceeding:
1. I have copied the database folder hatterastc from the server to 
my PC.(This database is embedded in 1 application.)

2. I get the error when I execute the following command:
ij connect 'jdbc:derby:hatterastc';
ERROR 08004: Connection refused : Invalid authentication.

I suppose the CALL command can be executed only after connecting to a 
database.

I have a Windows System and using the ij prompt from the cmd.
Please let me know how to reset the password from the ij prompt.


Hi Sonny,

Can you please specify which version of Derby you are using? I.e., is it 
10.1 or maybe 10.5?

(10.0 was released in 2004, 10.6 in 2010)

Now, resetting the password isn't supposed to be easy, as it would 
defeat the purpose of authentication.
Except from connecting as the system user (which may very well be the 
user you have forgotten the credentials for), I'm not aware of any way 
to reset the password for a user for which you don't have the password.


Unless the system security has been configured using derby.properties 
(which isn't recommended because it is a weak mechanism), I don't know 
how to help.



Regards,
--
Kristian



Thanks again.

Regards,
Sonny Laskar
India


-Thomas thomas.k.h...@t-online.de wrote: -

To: derby-user@db.apache.org
From: Thomas thomas.k.h...@t-online.de
Date: 11/29/2010 01:23PM
Subject: Re: ERROR 08004: Connection refused : Invalid authentication.

Sonny Laskar sonny.las...@... writes:



 Dear All,I am new to Derby.I have forgotten the
username/password set for
connecting to my derby database.Now when I tried to connect , i
get the invalid
authentication error (ERROR 08004: Connection refused : Invalid
authentication.)Please let me know if I can reset this password or
if I can
create a new user.Also I want to know that if I want to create the
clone of a
derby database , can I just copy the folder and use
it?Regards,Sonny LaskarIndia
 =-=-=Notice: The information contained in this
e-mailmessage and/or attachments to it may contain confidential or
privileged
information. If you are not the intended recipient, any
dissemination, use,
review, distribution, printing or copying of the information
contained in this
e-mail message and/or attachments to it are strictly prohibited.
If you have
received this communication in error, please notify us by reply
e-mail or
telephone and immediately and permanently delete the message and any
attachments. Thank you


login in with your data base owner account you can reset the
password of a user
by calling
SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY('derby.user.username' with
an empty password as the second parameter passed. Assuming you are
using the
builtin authentication provider. With the same stored procedure
you can also
create new users. (see reference guie on page 327 for details on
this system
built-in stored procedure)

To your second question: Yes, to clone a database you can just
copy the folder.



=-=-=
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you

Re: does Derby honor the statement.setQueryTimeout(int) ?

[Kristian Waagan]

 On 20.10.10 17:16, florin.herin...@sungard.com wrote:


Hi everybody,

I’m using Derby latest 10.6.2.1 and I’m having a problem with 
statement.setQueryTimeout(int).


Namely I want to set a small value for the timeout (5 seconds) so that 
the statement (an update statement) will fail (relatively) fast if 
some other transaction is holding a write lock on the same row.


The problem is that setQueryTimeout seems to be ignored, and the 
update is waiting until the global timeout - 60+ seconds.


Is that a known problem with Derby ? Or am I doing something wrong ?



Hi Florian,

I'm not sure, but if Derby is waiting for a lock, it may not honor the 
query timeout.
If the statement is doing normal processing, the query timeout should 
take effect. If your application code is set up correctly for retrying, 
is it an option to lower the lock timeout as well?



Regards,
--
Kristian


Best Regards,

Florin Herinean

Re: AW: does Derby honor the statement.setQueryTimeout(int) ?

[Kristian Waagan]

 On 20.10.10 17:38, florin.herin...@sungard.com wrote:

Hi Kristian,

Changing of the global timeout is not an option, since I need to have most of the 
statements behave normally (to wait as long as neccessary to succeed) and just a few 
statements with a fail fast behavior.


I don't know if it is an option, but maybe it is possible to change the 
lock timeout value (and possibly the deadlock timeout) if the query 
timeout is smaller than the lock timeout value.

Is doing this acceptable in principle?
Does anyone know that piece of the code well enough to say if that's a 
viable improvement?



Just a thought,
--
Kristian


The concrete problem: I have a kind of a timer that fires events at regular 
intervals (5 sec) and I have to perform some actions. It might be possible 
under certain circumstances that the performing of the actions take (much) 
longer than the fixed rate of the timer, so that the next events are fired 
before the long running one is finished. However, if the actions are involving 
the same sets of rows, I want that the overlapping ones to fail fast and roll 
back the transaction.

Now because Derby is waiting indefinitely, this scenario (events are comming 
via jms messages) will lead to the consumming of all available database 
connections, since for each jms message a new thread/transaction is created and 
implicitely a new connection.

Regards,

Florin

-Ursprüngliche Nachricht-
Von: Kristian Waagan [mailto:kristian.waa...@oracle.com]
Gesendet: Mittwoch, 20. Oktober 2010 17:24
An: derby-user@db.apache.org
Betreff: Re: does Derby honor the statement.setQueryTimeout(int) ?


   On 20.10.10 17:16, florin.herin...@sungard.com wrote:

Hi everybody,

I'm using Derby latest 10.6.2.1 and I'm having a problem with
statement.setQueryTimeout(int).

Namely I want to set a small value for the timeout (5 seconds) so that
the statement (an update statement) will fail (relatively) fast if
some other transaction is holding a write lock on the same row.

The problem is that setQueryTimeout seems to be ignored, and the
update is waiting until the global timeout - 60+ seconds.

Is that a known problem with Derby ? Or am I doing something wrong ?


Hi Florian,

I'm not sure, but if Derby is waiting for a lock, it may not honor the query 
timeout.
If the statement is doing normal processing, the query timeout should take 
effect. If your application code is set up correctly for retrying, is it an option to 
lower the lock timeout as well?


Regards,
--
Kristian


Best Regards,

Florin Herinean

Re: cannot make connection

[Kristian Waagan]

 On 02.10.10 17:18, laishram rahul wrote:

Sir,
Im using eclipse Helios wtp latest release, connection with derby
embeddeb jdbc driver is working fine but i CANNOT make connection with
DERBY CLIENT JDBC DRIVER.
please provide me some help if it is possible...


Hello,

Are you connecting to a server on the same machine (localhost), or a 
remote host?
If the latter, note that the Derby network server will only allow 
connections from localhost by default.


It would also help if you post the exception you get when trying to connect.


Regards,
--
Kristian

Re: Problem inserting a record with derby 10.6.1.0

[Kristian Waagan]

 On 01.10.10 09:44, Mamatha Kodigehalli Venkatesh wrote:


Hello all,

I too upgraded derby from 10.5.3.0 to 10.6.1.0, I am getting below 
exception


When I tried to insert a record into the table using a prepared statement.

java.sql.SQLException: Java exception: 
'org.apache.derby.iapi.sql.execute.Result


SetFactory.getProjectRestrictResultSet(Lorg/apache/derby/iapi/sql/execute/NoPutR

esultSet;Lorg/apache/derby/iapi/services/loader/GeneratedMethod;Lorg/apache/derb

y/iapi/services/loader/GeneratedMethod;ILorg/apache/derby/iapi/services/loader/G

eneratedMethod;IZZDD)Lorg/apache/derby/iapi/sql/execute/NoPutResultSet;: 
java.la


ng.NoSuchMethodError'.

at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknow


n Source)

at 
org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)


at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)

at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException


(Unknown Source)

at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Un


known Source)

at 
org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown So


urce)

at 
org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown So


urce)

at 
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown So


urce)

at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Un


known Source)

at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unkno


wn Source)

However Iam able to insert a record into table through ij editor.



Hi Mamatha,

Looks like Derby is using some old generated code or something, and that 
the referred method no longer exists. If that's the case, there is a bug 
in the Derby upgrade logic.


Did you boot the database with upgrade=true, or did you do a soft 
upgrade [1]?
Also, I'm assuming that you ran with the released Derby jars and not a 
custom build.



Regards,
--
Kristian

[1] http://wiki.apache.org/db-derby/UpgradingTen


Thanks

Mamatha



*From:* Lily Wei [mailto:lily...@yahoo.com]
*Sent:* Tuesday, September 21, 2010 1:49 AM
*To:* Derby Discussion
*Cc:* avinash.b...@nsn.com
*Subject:* Re: Problem!

Hi Avinash:
   You can find the changes from 10.5.3.0 to 10.6.1.0 in here: 
http://wiki.apache.org/db-derby/DerbyTenSixOneRelease


   Does your application change behavior because of new features? As 
Kristian point out, the trace will give us more information in turn of 
which part of Derby the problem occurs along with what operation is 
performed to upgrade derby from 10.5.3.0 to 10.6.1.0.



Hope this help,
Lily



*From:* Kristian Waagan kristian.waa...@oracle.com
*To:* derby-user@db.apache.org
*Sent:* Fri, September 17, 2010 3:47:21 AM
*Subject:* Re: Problem!

On 17.09.10 12:11, Bhat, Avinash (NSN - IN/Bangalore) wrote:

 Hi,

 This is avinash bhat, I am facing a problem in derby upgrade. 
Currently I have 10.5.0.3 version installed in my system and when I 
tried upgrading it to 10.6.0.1 my software is not able to load it is 
throwing null pointer exception. But it is able to create databases 
for the first time and if database already exist it says Null pointer 
exception.


 Can I know what is the problem which I am facing? Or do I know what 
major difference that you people have made between 10.5.0.3 and 10.6.0.1?



Hi Avinash,

I don't know why you get this error.
Can you post the stack trace you get when hitting the problem?
( written to the console, or to derby.log )
Also, are you doing a hard upgrade (specifying upgrade=true in the 
connection URL), or are you doing a soft upgrade (no special 
attributes in the connection URL)?


The trace may tell us in which part of Derby the problem occurs.


Regards,
-- Kristian

 With regards,
 Avinash Bhat

Re: Problem inserting a record with derby 10.6.1.0

[Kristian Waagan]

 On 01.10.10 10:11, Bhat, Avinash (NSN - IN/Bangalore) wrote:

Hi We /Kristian ,
our application supports SHA 1 as a default but the default in derby 
10.6.1.0 is SHA 256. I Tried changing that to the supporting algorithm
by out application. It Fine now. Working fine. Thanks for your kind 
information and help.


Hi Avinash,

Great to hear that it is working for you now.

However, Derby should not fail with a NullPointerException. If you still 
have the stack trace laying around, do you mind posting it to the list?
(we only need the lines referring to Derby code, see also 
http://wiki.apache.org/db-derby/UnwindExceptionChain )



Regards,
--
Kristian


**
*Best Regards,*
*Avinash V.Bhat| Nokia Siemens Networks | SPA2* |( +91 (80) 4363 
2091(Direct) | ( (858) 2091(VOIP) | y : avinash.b...@nsn.com



*From:* ext Mamatha Kodigehalli Venkatesh 
[mailto:mamatha.venkat...@ness.com]

*Sent:* Friday, October 01, 2010 1:14 PM
*To:* Derby Discussion; derby-user-i...@db.apache.org; 
derby-user-...@db.apache.org
*Cc:* Bhat, Avinash (NSN - IN/Bangalore); kristian.waa...@oracle.com; 
lily...@yahoo.com

*Subject:* RE: Problem inserting a record with derby 10.6.1.0

Hello all,

I too upgraded derby from 10.5.3.0 to 10.6.1.0, I am getting below 
exception


When I tried to insert a record into the table using a prepared statement.

java.sql.SQLException: Java exception: 
'org.apache.derby.iapi.sql.execute.Result


SetFactory.getProjectRestrictResultSet(Lorg/apache/derby/iapi/sql/execute/NoPutR

esultSet;Lorg/apache/derby/iapi/services/loader/GeneratedMethod;Lorg/apache/derb

y/iapi/services/loader/GeneratedMethod;ILorg/apache/derby/iapi/services/loader/G

eneratedMethod;IZZDD)Lorg/apache/derby/iapi/sql/execute/NoPutResultSet;: 
java.la


ng.NoSuchMethodError'.

at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknow


n Source)

at 
org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)


at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)

at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException


(Unknown Source)

at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Un


known Source)

at 
org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown So


urce)

at 
org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown So


urce)

at 
org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown So


urce)

at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Un


known Source)

at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unkno


wn Source)

However Iam able to insert a record into table through ij editor.

Thanks

Mamatha



*From:* Lily Wei [mailto:lily...@yahoo.com]
*Sent:* Tuesday, September 21, 2010 1:49 AM
*To:* Derby Discussion
*Cc:* avinash.b...@nsn.com
*Subject:* Re: Problem!

Hi Avinash:
   You can find the changes from 10.5.3.0 to 10.6.1.0 in here: 
http://wiki.apache.org/db-derby/DerbyTenSixOneRelease


   Does your application change behavior because of new features? As 
Kristian point out, the trace will give us more information in turn of 
which part of Derby the problem occurs along with what operation is 
performed to upgrade derby from 10.5.3.0 to 10.6.1.0.



Hope this help,
Lily



*From:* Kristian Waagan kristian.waa...@oracle.com
*To:* derby-user@db.apache.org
*Sent:* Fri, September 17, 2010 3:47:21 AM
*Subject:* Re: Problem!

On 17.09.10 12:11, Bhat, Avinash (NSN - IN/Bangalore) wrote:

 Hi,

 This is avinash bhat, I am facing a problem in derby upgrade. 
Currently I have 10.5.0.3 version installed in my system and when I 
tried upgrading it to 10.6.0.1 my software is not able to load it is 
throwing null pointer exception. But it is able to create databases 
for the first time and if database already exist it says Null pointer 
exception.


 Can I know what is the problem which I am facing? Or do I know what 
major difference that you people have made between 10.5.0.3 and 10.6.0.1?



Hi Avinash,

I don't know why you get this error.
Can you post the stack trace you get when hitting the problem?
( written to the console, or to derby.log )
Also, are you doing a hard upgrade (specifying upgrade=true in the 
connection URL), or are you doing a soft upgrade (no special 
attributes in the connection URL)?


The trace may tell us in which part of Derby the problem occurs.


Regards,
-- Kristian

 With regards,
 Avinash Bhat

Re: Problem!

[Kristian Waagan]

 On 17.09.10 12:11, Bhat, Avinash (NSN - IN/Bangalore) wrote:


Hi,

This is avinash bhat, I am facing a problem in derby upgrade. 
Currently I have 10.5.0.3 version installed in my system and when I 
tried upgrading it to 10.6.0.1 my software is not able to load it is 
throwing null pointer exception. But it is able to create databases 
for the first time and if database already exist it says Null pointer 
exception.


Can I know what is the problem which I am facing? Or do I know what 
major difference that you people have made between 10.5.0.3 and 10.6.0.1?




Hi Avinash,

I don't know why you get this error.
Can you post the stack trace you get when hitting the problem?
( written to the console, or to derby.log )
Also, are you doing a hard upgrade (specifying upgrade=true in the 
connection URL), or are you doing a soft upgrade (no special attributes 
in the connection URL)?


The trace may tell us in which part of Derby the problem occurs.


Regards,
--
Kristian


With regards,
Avinash Bhat

Re: question: is is possible to identify open resources?

[Kristian Waagan]

 On 11.09.10 10:16, Patrick Holthuizen wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

I use Derby with an in-memory database for unit testing. All unit
tests share a single connection. Now I want to assert (not close) (in
the tear down of a unit test) that all result sets and statements used
during the test are properly closed. Is it possible to obtain all
resources related to a connection which are currently open?


Hi Patrick,

Not that I'm aware of.
The only thing I can think of is issuing a rollback/commit, but that 
doesn't guarantee that resources are closed. In the Derby test framework 
we have utility methods keeping track of the resources, and they are 
closed automatically (if not closed already) in the tear down method.


Is there a specific reason why you don't want to close the connection?


Regards,
--
Kristian


Thank you in advance.

Sincerely,
Patrick Holthuizen
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkyLOuEACgkQ2xeQKUlEiTbIlwCcDMlqnzmuVpnbERGKUOYRC3LL
tMIAniDkMI/pO3OAuwEQKUuenwI8WFRl
=Qc/F
-END PGP SIGNATURE-

Re: Reclaiming space from Derby

[Kristian Waagan]

 On 08.09.10 12:50, Paul Taylor wrote:
HI, I use derby in embedded mode, I let the user specify a maxmium 
size for the derby database files, my program monitors this and if it 
goes over the size I use SQL to delete records that are no longer 
necessary in the hope of shrinking the database files size, but it 
doesn't, how can I force Derby to shrink back down ?


Hi Paul,

You have to run the compress system procedures. See these links for more 
information:

http://db.apache.org/derby/docs/dev/ref/ref-single.html#rrefaltertablecompress
http://db.apache.org/derby/docs/dev/ref/ref-single.html#rrefproceduresinplacecompress


Hope this helps,
--
Kristian



thanks Paul

Re: Create trigger with 2 sql statements

[Kristian Waagan]

 On 23.08.10 11:52, Andrzej Kasprzyszyn wrote:

Hi All

I would like to create following trigger

CREATE TRIGGER COUNTRY_TRG
AFTER INSERT ON COUNTRY
DELETE FROM aTable WHERE table_name = 'country'
INSERT INTO aTable(table_name, msg) VALUES('country', current_timestamp)


I am receiving syntax error:

  11:40:55  [CREATE - 0 row(s), 0.000 secs]  [Error Code: -1, SQL
State: 42X01]  Syntax error: Encountered INSERT at line 4, column 1.
... 1 statement(s) executed, 0 row(s) affected, exec/fetch time:
0.000/0.000 sec [0 successful, 0 warnings, 1 errors]


Is it possible that Derby doesn't allow two sql statements in triggers action?


Hi Andrzej,

Yes, you can only specify a single SQL statement. If you want to do 
more, I think you have to write (and make available) a Java stored 
procedure and use CALL PROCEDURE in your trigger.



Regards,
--
Kristian


Thanks in advance,
Andrzej

Re: Java exception: '-1: java.lang.ArrayIndexOutOfBoundsException' error

[Kristian Waagan]

On 13.08.10 12:45, Shankar Devi wrote:

Hi,
Please let me know the timeframe for when this issue is likely to be
fixed.
   


Hi,

I'm not sure we have identified the real problem yet.
Can you answer my latest question which I believe was:
-
 1. It looks like once connection is selecting from the BLOB, and then
 another connection
 deletes (or updates) it. Does this sound plausible for your application?
 [Devi] - yes.It is possible.


In that case, how do you obtain the value?
  a) InputStream is = rs.getBinaryStream(...);
  b) Blob blob = rs.getBlob(...) ; InputStream is = blob.getBinaryStream()

-

It is impossible to say when this issue will be fixed, because:
 o nobody has committed to fix it
 o there is no runnable repro to demonstrate the bug
 o there is still no *clear* understanding of what the underlying 
problem is (answering the question might help, providing a repro would 
help for certain )


I do however have some thoughts, and I do plan to follow up on those.
In any case, the next 10.6 maintenance release hasn't been scheduled 
yet. You could of course roll your own build once the fix has been 
committed to the source repository.



Regards,
--
Kristian


Thanks  Regards,
Devi.

   

Re: Can't find database

[Kristian Waagan]

On 12.08.10 01:07, David Van Couvering wrote:
Looks like it was a slash/forward-slash issue.  I changed my code to 
use File.separator and this very strange bug (it would only happen 
sometimes, and only on some machines) looks like it has gone away.


Since the problem is intermittent, could it be caused by multiple 
threads trying to boot the database?
And in case somebody wants to pursue this issue later, do you know if it 
happened across a range of different JVMs?



Thanks,
--
Kristian



Thanks,

David

On Tue, Aug 10, 2010 at 1:48 AM, Kristian Waagan 
kristian.waa...@oracle.com mailto:kristian.waa...@oracle.com wrote:


On 09.08.10 23:55, David Van Couvering wrote:

I am getting these intermittent errors that I can't reproduce on
my machine:

Database 
'C:\VontuDev\main\dist\scan\incremental_index/MONITOR/INCREMENTAL_INFO' not 
found

I read the docs, and there is no discussion about whether a mixed
\ / path is a problem on some systems.  Could it be?  Do I
need to normalize the path to all forward slashes?


Hi David,

I guess there can be problems related to mixing different path
separators, but testing on OpenSolaris does suggest that the
monitor and the in-memory back end are capable of handling it:

java
-Dderby.system.home=C:\VontuDev\main\dist\scan\incremental_index
-cp classes org.apache.derby.tools.ij
ij version 10.7
ij connect 'jdbc:derby:memory:MONITOR/INCREMENTAL_INFO;create=true';
ij

Then, from derby.log:

memory:/home/user/derby/wdir/trunk-netbeans/C:\VontuDev\main\dist\scan\incremental_index/MONITOR/INCREMENTAL_INFO

However, trying this with an on-disk database fails. My guess is
because I'm on UNIX, and then the colon from c:\ shows up in a
place it's not supposed to be.
In general, I think your mileage depends on what java.io.File is
capable of, but there might be problems in the explicit path
handling within Derby.

Do you get an assert failure if you run with a sane build when you
try to boot the database?


Regards,
-- 
Kristian






What can cause this error to happen?  I checked, and the
directory is there on the system.

Thanks,

David

-- 
David W. Van Couvering


http://www.linkedin.com/in/davidvc
http://davidvancouvering.blogspot.com
http://twitter.com/dcouvering





--
David W. Van Couvering

http://www.linkedin.com/in/davidvc
http://davidvancouvering.blogspot.com
http://twitter.com/dcouvering

Re: Log connects and disconnects

[Kristian Waagan]

On 12.08.10 10:44, Damian Hofmann wrote:

Thanks Kristian

I used the Thread-View of visualvm, where I could see the states of 
DRDAConnectionThreads. This showed me, that my bugfix had improved the 
situation, but a small problem still remained.


Hi,

I see.
You might also find the attributes AccumulatedConnectionCount and 
ActiveConnectionCount useful (these are published by the NetworkServer 
MBean). If the active count equals the accumulated count after keeping 
the application running for a while, the connections aren't closed. Even 
though sockets will be closed for instance when the JVM exits, it is 
better (and recommended) to close them sooner if you know the connection 
won't be used again.


There are more connection counts exposed by the MBean, and they're 
slightly confusing unless you know about the derby.drda.maxThreads and 
derby.drda.timeSlice properties...




I also filed an improvement request:
https://issues.apache.org/jira/browse/DERBY-4769


Thanks,
--
Kristian



Greets
Damian



Am 11.08.10 11:31, schrieb Kristian Waagan:

On 11.08.10 10:42, Damian Hofmann wrote:

Hi Derby Community

Short question: Are disconnects supposed get logged if the option 
derby.drda.logConnections is enabled? If not, how can I see if the 
connections get properly closed?


Hi Damian,

I don't think disconnects are logged. I see two things you can do 
with Derby currently:

 o enable client side logging
 o use JMX and look at the information provided by NetworkServer 
MBean [1]


I'm thinking the JMX approach is the easiest way to get a picture of 
the situation in your network server (could use jconsole or visualvm).
If you would like to see logging of disconnects to derby.log, you 
should file a request in Jira [2]. I expect the required changes to 
be small, perhaps only adding tracing code to either 
DRDAConnThread.closeSession() or Session.close().



Regards,

Re: Derby Self-study Tutorial

[Kristian Waagan]

On 10.08.10 21:41, Sal Ishmel wrote:

  May someone please assist me in this:

1.  Can i do the Derby Self-study Tutorial on Mac OS 10.6?
2.  I am running Eclipse 3.6 and Netbeans 6.9, both bundled with the Apache 
Derby.
Can I run the tutorials on these IDEs? Thanx in advance.
   


Hi Sal,

I'm not aware of any issues that would suggest you cannot do what you 
describe.
Although the tutorial is using ij from the command line / shell, it 
should be possible to use the SQL facilities of your IDE instead.



Good luck :)
--
Kristian




Sal

Re: Log connects and disconnects

[Kristian Waagan]

On 11.08.10 10:42, Damian Hofmann wrote:

Hi Derby Community

Short question: Are disconnects supposed get logged if the option 
derby.drda.logConnections is enabled? If not, how can I see if the 
connections get properly closed?


Hi Damian,

I don't think disconnects are logged. I see two things you can do with 
Derby currently:

 o enable client side logging
 o use JMX and look at the information provided by NetworkServer MBean [1]

I'm thinking the JMX approach is the easiest way to get a picture of the 
situation in your network server (could use jconsole or visualvm).
If you would like to see logging of disconnects to derby.log, you should 
file a request in Jira [2]. I expect the required changes to be small, 
perhaps only adding tracing code to either DRDAConnThread.closeSession() 
or Session.close().



Regards,
--
Kristian

[1] For testing it should be pretty simple to get started: just connect 
to the network server process locally with 
jconsole/visualvm/your_favorite_jxm_tool. In deployment you may need 
to do some more work (see for instance 
http://wiki.apache.org/db-derby/DerbyJMXQuickStart), especially for 
remote access as well.

[2] https://issues.apache.org/jira/browse/DERBY



Background:
I recently had a problem with one of our Applications not responding 
anymore. I suspected that the problem was with the derby network 
server blocking because too many connections had been opened. In the 
logfile tousands of connections are logged, but not a single 
disconnect. So after I changed some code I excepted to see some 
disconnects. But the logfile still only shows connects.


Now I do not know if ...
a) ... there has never been a problem with to many connections and I 
have solved the wrong problem

b) ... my solution doesn't work
c) ... there was a problem with connections, my solution works, but 
the logfile simply doesn't tell me that


Thanks for your help
Damian

Re: Can't find database

[Kristian Waagan]

On 09.08.10 23:55, David Van Couvering wrote:
I am getting these intermittent errors that I can't reproduce on my 
machine:


Database 
'C:\VontuDev\main\dist\scan\incremental_index/MONITOR/INCREMENTAL_INFO' not 
found

I read the docs, and there is no discussion about whether a mixed \ 
/ path is a problem on some systems.  Could it be?  Do I need to 
normalize the path to all forward slashes?


Hi David,

I guess there can be problems related to mixing different path 
separators, but testing on OpenSolaris does suggest that the monitor and 
the in-memory back end are capable of handling it:


java -Dderby.system.home=C:\VontuDev\main\dist\scan\incremental_index 
-cp classes org.apache.derby.tools.ij

ij version 10.7
ij connect 'jdbc:derby:memory:MONITOR/INCREMENTAL_INFO;create=true';
ij

Then, from derby.log:
memory:/home/user/derby/wdir/trunk-netbeans/C:\VontuDev\main\dist\scan\incremental_index/MONITOR/INCREMENTAL_INFO

However, trying this with an on-disk database fails. My guess is because 
I'm on UNIX, and then the colon from c:\ shows up in a place it's not 
supposed to be.
In general, I think your mileage depends on what java.io.File is capable 
of, but there might be problems in the explicit path handling within Derby.


Do you get an assert failure if you run with a sane build when you try 
to boot the database?



Regards,
--
Kristian




What can cause this error to happen?  I checked, and the directory is 
there on the system.


Thanks,

David

--
David W. Van Couvering

http://www.linkedin.com/in/davidvc
http://davidvancouvering.blogspot.com
http://twitter.com/dcouvering

Re: Low performance when execute more then 100 inserts

[Kristian Waagan]

On Tue, Aug 03, 2010 at 04:01:56PM -0300, Fabricio Pedroso Jorge wrote:
 Hello,
 
 I have a Java application that reads a .txt file, validates the informations
 and then executes several inserts into de Derby 10.5 database. The problem is
 that when i have a lot of inserts (more than 1000), the execution of these
 inserts gets slower and slower until it gets unsupportable for the user to
 wait.
 
 I made a test, inserting the same lines of the file into an Oracle 10g
 database, and it worked pretty well.
 
 So, is there any configuration, or otimization on Derby that i can perform to
 make it work???

Hi Fabrício,

It sounds strange that inserting a few thousands lines should be such of
a problem that the database grinds to a halt.

I only have a few general questions, hopefully the answers can give us a
clue of what's going on.

 1. Are you using the embedded or the client/server configuration?
 2. How big is the file being imported?
 3. Is autocommit on, or are you committing at intervals?
 4. Is there a primary/unique key on the table? Any other indexes?
 5. If possible, can you run your application until it slows down badly,
and then run jstack on the process?
(could also use visualvm or similar to obtain the stack trace)
 6. Are you running with default Derby and JVM settings?
(and, can you confirm that it isn't garbage collection that slows
 down the application)


Regards,
-- 
Kristian

 
 Thanks for the attention
 
 --
 Atenciosamente,
 
 Fabrício Pedroso Jorge.
 
 Administrador de Banco de Dados Trainee - SEFA-PA
 Bacharel em Ciência da Computação - CESUPA
 
 Resumo Profissional (Professional Skills):
 http://br.linkedin.com/pub/fabricio-jorge/19/554/58b

Re: Shutting down a database that is already shut down

[Kristian Waagan]

On Fri, Jul 30, 2010 at 02:30:34PM -0700, David Van Couvering wrote:
 Do I have to worry about ensuring that a database is running before I try to
 shut it down, or is it OK to get a connection with URL ;shutdown=true on a
 database that's not booted.
 
 If I were to do that, does it boot the database and then shut it down again, 
 or
 does it do nothing?

Hi David,

Derby won't boot the database, but you will get a database not found
exception. The exception may also be written to derby.log, which casues
the file to grow a lot faster if you issue a lot of shutdown requests
for non-booted databases.

You get the exception because the getConnection call must either
return a valid connection or throw an exception. Since we haven't shut
down the database (it wasn't booted), we cannot throw an exception
saying we did so. Getting database not found seems a bit odd too, but
maybe Derby currently has to boot a database to confirm that it exists?


Regards,
-- 
Kristian

 
 Thanks,
 
 David
 
 --
 David W. Van Couvering
 
 http://www.linkedin.com/in/davidvc
 http://davidvancouvering.blogspot.com
 http://twitter.com/dcouvering

Re: backing up the DB

[Kristian Waagan]

On Mon, Aug 02, 2010 at 04:00:41AM -0700, vadali wrote:
 
 Hello,
 
 I have a few tables (with FK), that i need to save and restore. 
 My original plan was to save the entire directory of the database on one
 machine, then reload it on another machine.
 
 The problem is that i might want to only save a portion of that db (say only
 lines in tables that are newer then a certain date - i do keep the date for
 each line).
 

Hi Vadali,

What are your requirements for uptime on the primary db?
Do you have to use online backup, or can you block update transactions
for a while?

 whats my best method? 
 
 I thought about manually coping all the interesting lines from one db to
 another one (lets name it backupdb), but that could be slow.. 
 another option i thought, would be to copy the entire db to backupdb, then
 removing all the lines i dont need, but when i tried that, the size of the
 backupdb directory was even larger then the original...

This is probably because Derby isn't releasing unused space to the OS
before it is told to do so. You should run the compress routines on the
tables from which you deleted a large number of rows.
The extra space may be log records, which will be deleted after a checkpoint / 
clean shutdown.

 
 Do you have any better options?

Seems to me you have to decide if you want to filter the data up front
or after you have created the backup.
In the former case you could write your own database copy using SQL to
filter the data (or maybe write a VTI), in the latter case you would
have to account for the time spent deleting unwanted data and then
compressing the tables to save space.

It might also help people to come up with better answers if you tell us
roughly how large you expect your database to be.


Regards,
-- 
Kristian

 
 best regards,
 vadali
 -- 
 View this message in context: 
 http://old.nabble.com/backing-up-the-DB-tp29324036p29324036.html
 Sent from the Apache Derby Users mailing list archive at Nabble.com.
 

Re: Database-side programming

[Kristian Waagan]

On 18.07.10 18:15, Saleem EDAH-TALLY wrote:

Hello,

This concerns getting information from 'inside' Derby through JAVA code called
by a procedure.

The nested connection can be obtained with jdbc:default:connection. However
the URL attributes cannot be retrieved in any way.


Hi,

Maybe it would help if you tell why you want to do this, so people can 
suggest a different mechanism if appropriate?





conn.getClientInfo() returns an empty Properties object.
   


Adding support for getClientInfo is briefly discussed in DERBY-3849 [1]. 
Would it be helpful in your use case?



conn.getMetaData().getUserName() returns nothing.
   


Can you obtain the current user name with CURRENT_USER?


The Derby Developper's Guide states the following :

*
Connection conn = DriverManager.getConnection(
 jdbc:default:connection);
URL attributes are not supported as part of this connection URL. Any URL
attributes specified in a Properties object, user name, or password that are
passed to a java.sql.DriverManager.getConnection() call will be ignored.
*

I don't want to specify URL attributes. I wish to know how can one retrieve
information about these current URL attributes, namely, SSL mode.
   


I'm not aware of a way to determine if the connection to the database is 
encrypted from within a stored procedure.
Doing so seems a bit out of spec, which is why I asked what you want 
to achieve :)



Regards,
--
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-3849


Also, how can one determine in database-side programming if it's the client
driver or the embedded driver that's in use ? i.e, without parsing the URL
(presence of // would mean client driver in use, else it's embedded driver).

Thank you.


   

Re: How to best constrain database space with many records being inserted and deleted

[Kristian Waagan]

On 06.07.10 20:09, Bergquist, Brett wrote:


I  have a situation where my system is adding about 600K records a day 
to a table and also deleting about 600K records a day.  Basically it 
is keeping 30 days worth of history of some network management service 
level agreement data.  So each day about 600K new tests are performed 
and recorded in the database and each day after the 30 day mark, about 
600K old records are purged out.  On average there is about 18 million 
records in the table.




Hi Brett,

Are you able to determine if the growing conglomerate is a heap or a btree?
Do to that, find the largest file(s) in the seg0 directory of your 
database. Convert the number from hex to dec, i.e. 'c300.dat' - 768.

Then do something like:
select isindex, descriptor from sys.sysconglomerates where 
conglomeratenumber = ?;

(? would be 768 for the example file)


Regards,
--
Kristian

I have little to no down time for database maintenance.  Maybe 2 hours 
per month maximum.  What I am seeing is that the database is growing 
and it does not seem to be reusing the deleted space.  Should it be?  
The records being inserted are exactly the size of the records being 
deleted.


I know that I could use the SYSCS_UTIL.SYSCS_COMPRESS_TABLE to reclaim 
the space, but I am not so interested in giving back to the OS, but 
rather ensuring the space available from the deleted records is 
reused.  I have attempted to reclaim the space with this command, 
however, and about 5 hours of time is just too much.


I also see there is a SYS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE but I am 
not sure what the PURGE_ROW option is going to do.  It sounds like it 
is something that I want to look into, but the documentation is not 
clear enough if that is what I need.


Thanks in advance.

Brett //

Re: Cannot setup in-memory embedded derby

[Kristian Waagan]

On 15.07.10 11:37, Sergey Ponomarev wrote:

Hi, help please.

I'm trying to use in-memory derby, but get database 'login' not 
found error (java.sql.SQLException: Error in allocating a connection. 
Cause: Connection could not be allocated because: База данных 'login' 
не найдена.)


I've included derby,jar to classpath, specified connection URL 
jdbc:derby:memory:login;create=true, can't understand what am i 
doing wrong.


create=true, so i suggest it should create and find it, but it 
doesn't. Did not find anything useful in google.


Hi Sergey,

I'm not sure, but I think your data source definition is invalid.
Probably DatabaseName takes precedence over URL (I'm not even sure Derby 
will use the URL property?).

If you remove URL, you should be able to add the following:

property name=Create value=create /
property name=DatabaseName value=memory:login /

Alternatively, if the create property doesn't work, you can maybe use 
this instead:

property name=ConnectionAttributes value=create=true /


Hope this helps,
--
Kristian



Here is the configuration:
domain.xml
===
jdbc-connection-pool 
datasource-classname=org.apache.derby.jdbc.EmbeddedDataSource 
res-type=javax.sql.DataSource name=TestPool

property name=URL value=jdbc:derby:memory:login;create=true /
property name=driverClass 
value=org.apache.derby.jdbc.EmbeddedDriver /

property name=DatabaseName value=login /
property name=User value= /
property name=Password value= /
/jdbc-connection-pool
jdbc-resource pool-name=TestPool jndi-name=jdbc/loginService /
=

persistence.xml
=
?xml version=1.0 encoding=UTF-8?
persistence version=2.0 
xmlns=http://java.sun.com/xml/ns/persistence; 
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation=http://java.sun.com/xml/ns/persistence 
http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd;

persistence-unit name=LoginServicePU transaction-type=JTA
providerorg.hibernate.ejb.HibernatePersistence/provider
jta-data-sourcejdbc/loginService/jta-data-source
exclude-unlisted-classesfalse/exclude-unlisted-classes
properties
property name=hibernate.hbm2ddl.auto value=update/
/properties
/persistence-unit
/persistence
===

I'm a newbie, please don't blame me, any help is very appreciated. 
Thank you.

Re: Unknown page format at page Page(257,Container(0, 432592)

[Kristian Waagan]

On 08.07.10 18:03, Erick Lichtas wrote:


Hi everyone,

I have been using an embedded setup of derby (version 10.4.1.3) for 
quite some time.  It appears that my database has become corrupt for 
some reason, possibly related to 
https://issues.apache.org/jira/browse/DERBY-3725




Hi,

I'm afraid the only proper way out of this is to restore from a backup.

Another option is to post only the damaged page (or post the full 
conglomerate/db itself) for investigation. You could also modify only 
the format id and run SYSCS_CHECK_TABLE again, but bit-fiddling at this 
level really isn't recommended...
Also, performing such an investigation is a lot of work, so I'm not sure 
anyone will do it.



Regards,
--
Kristian


Stacktrace

_java.sql.SQLException_: Unknown page format at page 
Page(257,Container(0, 432592))


  at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown 
Source)


  at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown 
Source)


  at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)

  at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
Source)


  at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown 
Source)


  at 
org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)


  at 
org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)


  at 
org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown 
Source)


  at 
org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source)


  at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source)

  at com.linoma.derby.corrupt.Derby.verifyDatabase(_Derby.java:108_)

  at com.linoma.derby.corrupt.DerbyMain.main(_DerbyMain.java:17_)

Caused by: _java.sql.SQLException_: No class was registered for 
identifier 0.


  at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown 
Source)


  at 
org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)


  at 
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown 
Source)


  ... 9 more

Caused by: ERROR XBM0U: No class was registered for identifier 0.

  at 
org.apache.derby.iapi.error.StandardException.newException(Unknown Source)


  at 
org.apache.derby.impl.services.monitor.BaseMonitor.classFromIdentifier(Unknown 
Source)


  at 
org.apache.derby.impl.services.monitor.BaseMonitor.newInstanceFromIdentifier(Unknown 
Source)


  at 
org.apache.derby.iapi.services.monitor.Monitor.newInstanceFromIdentifier(Unknown 
Source)


  at 
org.apache.derby.impl.store.raw.data.CachedPage.changeInstanceTo(Unknown 
Source)


  at 
org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown 
Source)


  at 
org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown Source)


  at 
org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(Unknown 
Source)


  at 
org.apache.derby.impl.store.raw.data.FileContainer.getNextHeadPage(Unknown 
Source)


  at 
org.apache.derby.impl.store.raw.data.BaseContainer.getNextPage(Unknown 
Source)


  at 
org.apache.derby.impl.store.raw.data.BaseContainerHandle.getNextPage(Unknown 
Source)


  at 
org.apache.derby.impl.store.access.conglomerate.GenericScanController.positionAtNextPage(Unknown 
Source)


  at 
org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown 
Source)


  at org.apache.derby.impl.store.access.heap.HeapScan.next(Unknown 
Source)


  at 
org.apache.derby.iapi.db.ConsistencyChecker.checkTable(Unknown Source)


  at 
org.apache.derby.catalog.SystemProcedures.SYSCS_CHECK_TABLE(Unknown 
Source)


  at 
org.apache.derby.exe.acf81e0010x0129xb2c4xb230x0015ed900.e5(Unknown Source)


  at 
org.apache.derby.impl.services.reflect.DirectCall.invoke(Unknown Source)


  at 
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.doProjection(Unknown 
Source)


  at 
org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(Unknown 
Source)


  at 
org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown 
Source)


  ... 4 more

Is there any way to tell for sure if this issue is related to 
DERBY-3725?  If so, and I upgrade the database, is there a way to 
repair the existing database?  Is there any additional information I 
can provide to give further insight?


Thank you,

ERICK J. LICHTAS

Linoma Software

Software Engineer

p. 402.944.4242 x714

f. 402.944.4243

www.LinomaSoftware.com www.linomasoftware.com

www.GoAnywheremft.com www.goanywheremft.com



__ Information from ESET NOD32 Antivirus, version of virus 
signature database 5262 (20100708) __


The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

Re: Database corruption on crash during table creation

[Kristian Waagan]

On 06.07.10 08:23, David Van Couvering wrote:
Hi, all.  I seem to have a repeatable issue where my database is 
getting corrupted, and I can only assume it's happening when the table 
is getting created, because I know Derby is great on handling crashes 
during normal operation.


First of all - is it true that a Derby database can get corrupted if 
the VM goes down in the middle of a DML like CREATE TABLE?


What I'm seeing is that upon restart, when I try to access the 
database, I get the error unable to create database FOO: the 
directory already exists.  The directory contains what appears to be 
a Derby database, but apparently not enough of one.


Hi David,

It would be great if you can create a Jira and attach the contents of 
the database directory (or maybe just the output from find). Including a 
runnable repro would be even better :)

Also:
  o Can the issue be reproduced using ij only?
  o Is this reproducible when only a single process is 
accessing/modifying the database?


I believe there has been a few reports of the error message you 
mentioned, but so far no repro has been available.


Having a forceCreate option requires some thought, it would be 
unfortunate if Derby deleted (i.e. misuse or exploit) a bunch of 
user/system files. This wouldn't be such a big deal if everyone 
tightened the Java security policy for Derby, but I'm not sure that's 
the case...



Regards,
--
Kristian



What I'm hoping for, is there a forceCreate=true kind of option when 
opening a database that forces the creation of the database in the 
given directory even if something is already there?


Otherwise it appears my only option is to detect this specific error 
('directory already exists'), and then blow away what's there and try 
again.  That's a chunk of work I'd rather not do, so I'm hoping you've 
already addressed it (ahem :)).


Thanks,

David

--
David W. Van Couvering

http://www.linkedin.com/in/davidvc
http://davidvancouvering.blogspot.com
http://twitter.com/dcouvering

Re: Android

[Kristian Waagan]

On 23.06.10 15:03, Paul French wrote:


Anyone tried a derby database on android? I need to look at this 
shortly but if it is a definite “does not work” then you will save me 
some time.



Hi,

I think you're out of luck for now, at least if you want this to be an 
easy ride :)

See DERBY-4458 [1].


Regards,
--
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-4458


Thanks

Paul

Re: Hung thread after another thread is interrupted

[Kristian Waagan]

On 22.06.10 10:03, Kristian Waagan wrote:

On 22.06.10 08:46, Luke Quinane wrote:

Hi Kristian,

I've played around with the code a bit during the day and the 
attached patch seems to fix the problem where another thread is 
stuck. I've never really look at the Derby code before so I'm not 
confident that it is correct however it did seem to fix things for me.


Thanks Luke,

I think the person who has worked most on the lock system recently is 
around. Hopefully he'll have a look at your patch.
To me the patch looks okay, but I don't have the cycles to verify it 
at the moment. I will however apply it to trunk and run the regression 
tests.


FYI, I didn't see any failures in the regression tests when applying the 
patch to trunk (had to regenerate the patch for it to apply, see 
attached file).



--
Kristian



My suspicions about write operations getting interrupted were 
groundless in this case, so it seems you have hit a different issue.



Regards,


Index: java/engine/org/apache/derby/impl/services/locks/LockSet.java
===
--- java/engine/org/apache/derby/impl/services/locks/LockSet.java   
(revision 956795)
+++ java/engine/org/apache/derby/impl/services/locks/LockSet.java   
(working copy)
@@ -273,12 +273,18 @@
 
 forever:   for (;;) {
 
-byte wakeupReason = waitingLock.waitForGrant(actualTimeout);
-
+byte wakeupReason = 0;
 ActiveLock nextWaitingLock = null;
 Object[] deadlockData = null;
 
 try {
+try {
+wakeupReason = waitingLock.waitForGrant(actualTimeout);
+} catch(StandardException e) {
+nextWaitingLock = control.getNextWaiter(waitingLock, 
true, this);
+throw e;
+}
+
 boolean willQuitWait;
 Enumeration timeoutLockTable = null;
 long currentTime = 0;
Index: java/engine/org/apache/derby/impl/services/locks/ConcurrentLockSet.java
===
--- java/engine/org/apache/derby/impl/services/locks/ConcurrentLockSet.java 
(revision 956795)
+++ java/engine/org/apache/derby/impl/services/locks/ConcurrentLockSet.java 
(working copy)
@@ -460,12 +460,18 @@
 
 forever:   for (;;) {
 
-byte wakeupReason = waitingLock.waitForGrant(actualTimeout);
-
+byte wakeupReason = 0;
 ActiveLock nextWaitingLock = null;
 Object[] deadlockData = null;
 
 try {
+try {
+wakeupReason = waitingLock.waitForGrant(actualTimeout);
+} catch(StandardException e) {
+nextWaitingLock = control.getNextWaiter(waitingLock, 
true, this);
+throw e;
+}
+
 boolean willQuitWait;
 Enumeration timeoutLockTable = null;
 long currentTime = 0;

Re: Java exception: '-1: java.lang.ArrayIndexOutOfBoundsException' error

[Kristian Waagan]

On 02.06.10 10:35, Shankar Devi wrote:

Hi Kristian,
Thanks for the reply.
1. It looks like once connection is selecting from the BLOB, and then
another connection
deletes (or updates) it. Does this sound plausible for your application?
[Devi] - yes.It is possible.
   


In that case, how do you obtain the value?
 a) InputStream is = rs.getBinaryStream(...);
 b) Blob blob = rs.getBlob(...) ; InputStream is = blob.getBinaryStream();


2.  Does this bug cause database corruption?
   


No.


--
Kristian


Thanks  Regards,
Devi


   

Re: Hung thread after another thread is interrupted

[Kristian Waagan]

On 21.06.10 09:20, Luke Quinane wrote:

Hi all,

We've seen a problem today where we have several threads querying our 
database and when one gets interrupted the others are stuck waiting 
for a lock. Here is the stack trace for the stuck thread(s):


daemon prio=4
DefaultExecutorService-pool-1-thread-47 Id=98 WAITING on 
org.apache.derby.impl.services.locks.activel...@6e6f45a1

at java.lang.Object.wait(Native Method)
-  waiting on org.apache.derby.impl.services.locks.activel...@6e6f45a1
at java.lang.Object.wait(Object.java:485)
at 
org.apache.derby.impl.services.locks.ActiveLock.waitForGrant(ActiveLock.java:115)
at 
org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(ConcurrentLockSet.java:463)
at 
org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(ConcurrentLockSet.java:855)
at 
org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(AbstractPool.java:297)
at 
org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(ConcurrentPool.java:28)
at 
org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(RowLocking2nohold.java:89)
at 
org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java:520)
at 
org.apache.derby.impl.store.access.heap.HeapController.lockRow(HeapController.java:638)
at 
org.apache.derby.impl.store.access.btree.index.B2IRowLocking3.lockRowOnPage(B2IRowLocking3.java:309)
at 
org.apache.derby.impl.store.access.btree.index.B2IRowLocking3._lockScanRow(B2IRowLocking3.java:599)
at 
org.apache.derby.impl.store.access.btree.index.B2IRowLockingRR.lockScanRow(B2IRowLockingRR.java:105)
at 
org.apache.derby.impl.store.access.btree.BTreeForwardScan.fetchRows(BTreeForwardScan.java:305)
at 
org.apache.derby.impl.store.access.btree.BTreeScan.fetchNextGroup(BTreeScan.java:1585)
at 
org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:327)
at 
org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:282)
at 
org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
at 
org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:427)

-  locked org.apache.derby.impl.jdbc.embedconnectio...@445d374b
at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:371)
...

So is using Thread.interrupt() the correct way to interrupt a long 
running query? Currently our worker threads are using 
'java.util.concurrent.ThreadPoolExecutor' to run our queries and it 
calls 'interrupt()' when it shuts down.


Hi Luke,

Unfortunately, Derby isn't currently able to cope correctly with getting 
interrupted by Thread.interrupt(). Based on limited information, and as 
it stands today, I think your options are:

 a) code special shutdown logic for Derby
 b) do not invoke Thread.interrupt()
 c) force Derby to not use NIO (?)

This issue seems like something the Derby community has to address, as 
it is easily encountered using Java standard libraries (i.e. 
java.util.concurrent).
It is not clear to me why the lock system fails. Do you have a stack 
trace from the thread being interrupted?


Also, can you describe your requirements for the process issuing 
Thread.interrupt()?
I.e, do you want to shut down your application hard, or just to cancel 
a single running query?
You may find some helpful information in this thread (see especially the 
posts from Stephan and Knut): 
http://old.nabble.com/cancel-a-running-query--td19903183.html



Regards,
--
Kristian



This was with Derby 10.5.3.0 and JRE 1.6.0_19-b04 on Windows Server 2008.


Thanks in advance,


Luke Quinane

Re: can't delete my DB directory

[Kristian Waagan]

On 09.06.10 03:32, oldmhe wrote:


The OS is Windows XP.

Since I'm using the embedded driver, I don't think it's possible to check
what process is hanging on to the file.  I.e., I believe it's the same
process as my program.  When my program exits, I'm able to delete the
directory manually.


Hi,

It would still be useful to see which files the process has open handles 
to. If there is no tool readily available for doing this, a crude way 
would be to run the partly successful delete and then just post a 
listing of the files that are left in the database directories.




Regarding your last question, my program does shutdown explicitly (as shown
in my original post).


Yes, you are shutting down the Derby engine, but not the specific 
database explicitly.
An engine shutdown should shut down all booted databases, but it would 
be nice to rule out a bug in this area.



Regards,
--
Kristian



Since posting, I found a solution, but I don't understand why it works.

Below is a simplification of what the program does:

1. load the embedded driver
 set AutoCommit to false
 connect to the DB engine, and create a DB
 create tables
 load the tables with data, and commit all work

2. using SELECT, read some data records, and create an output file.

3. shutdown the DB engine

4. try to delete the DB directory (and all files and subdirectories)
 exit

With regard to my initial post, Step 4 fails to delete all the files and
directories (it's able to delete most of them).

However:

1.  If I omit Step 2 (the reading of the DB), Step 4 succeeds.

2.  Or, if I add xxx.commit() between Step 2 and Step 3, then Step 4
succeeds.

It seems that a commit() is needed even though Step 2 makes no change to the
DB.  This is contrary to my expectations for two reasons:

a)  Since Step 2 is a read-only operation, I don't see why commit() is
needed.

b)  Even if a commit() is needed, the shutdown shouldease all DB
resources (and not hang on to any files).

Any thoughts?


Kristian Waagan-4 wrote:


Hello,

What operating system are you using?
Are you able to use the operation system's proper tool to check which
process (if any) is hanging on to the file?
(i.e. pfiles or lsof)

Also, do you see the same behavior if you in addition shut down the
database explicitly?
(i.e. 'DriverManager.getConnection(jdbc:derby:myDB;shutdown=true);')


Regards,
--
Kristian

Re: can't delete my DB directory

[Kristian Waagan]

On 04.06.10 18:51, oldmhe wrote:

Prior to exiting, my Java program issues a shutdown, and then tries to delete
the DB directory (recursively), but is not able to completely delete the
directory.  Here's the code:

try {
DriverManager.getConnection(jdbc:derby:;shutdown=true);
} catch (SQLException eSQLException) {
if (
eSQLException.getErrorCode() == 5

XJ015.equals(eSQLException.getSQLState())
   ) {

// successful shutdown
} else
printSQLException(eSQLException);
}
removeDir(new File(dbnmString));// remove DB directory

(removeDir is my own function)

If removeDir() is called before connecting to the DB, it works fine.  But
the above call to removeDir() only deletes some of the files and directories
within dbnmString -- as if Derby is still accessing them (or has not closed
them).

I also tried dropping the DB just before the shutdown code, with the same
results.

Any ideas would be appreciated.
   


Hello,

What operating system are you using?
Are you able to use the operation system's proper tool to check which 
process (if any) is hanging on to the file?

(i.e. pfiles or lsof)

Also, do you see the same behavior if you in addition shut down the 
database explicitly?

(i.e. 'DriverManager.getConnection(jdbc:derby:myDB;shutdown=true);')


Regards,
--
Kristian



   

Re: strange behavior using embedded driver

[Kristian Waagan]

On 01.06.2010 10:42, Marco Ferretti wrote:

Hi
During some edge testing of my application I came across a strange 
behavior using an embedded derby driver on windows: if, once the 
database is started, the machine is given a different ip address ( eg 
: the network is no longer available or the ISP dynamically assigns 
you a different one ) the database is not reachable anymore.
Is this an expected behavior ? If so, what is the best practice to 
avoid this problem ?


Hi Marco,

I've never heard about this, but it shouldn't be too hard to test.
What do you mean by unreachable?
Are you doing something like this?
 - connect to the database
 - unplug network cable
 - obtaining another connection in the same JVM fails


Thanks,
--
Kristian



Connection specs :
driver : org.apache.derby.jdbc.EmbeddedDriver
url : jdbc:derby:/DERBYDB/database/codecs2;create=false
Bundle-Version: 10.5.300.802917

TIA

Marco

--
No Im sorrys, no apologies, no regrets. 

Re: java.sql.SQLException: No suitable driver

[Kristian Waagan]

On 28.05.2010 18:44, Marcin Krol wrote:

Hello everyone,

I'm trying to use Derby here as embedded database in our test 
automation framework (that deploys itself on the remote endpoint and 
executes some Java classes to test functionality of the product).


Code:

private void createDerbyConnection() throws ClassNotFoundException, 
SQLException

{
Class.forName(org.apache.derby.jdbc.EmbeddedDriver);
String dburl = jdbc:derby:DerbyDB;create=true;
derbyconn = DriverManager.getConnection(dburl);
derbyconn.setAutoCommit(false);
derbyst = derbyconn.createStatement();
}



Hi Marcin,

Sorry for this vey vague answer, but could this have to do with 
classloaders?

Is the framwork you are using doing anything explicit about classloaders?

There is some classloader sensitive code in DriverManager.
Not sure in which context I stumbled over this, but I may have worked 
around the issue by using the Class.forName() method that takes a 
ClassLoader argument. Now, if this is the problem and the proposed fix 
is sound, you just have to figure out which classloader to use when 
loading the driver class :)



Regards,
--
Kristian



Now, the weird thing is that when the same class is using Derby in 
standalone program (ran e.g. on Linux) *on the same host*, or in 
Eclipse on my laptop, it all runs fine - I can connect to the db and 
operate on it all right.


It's only when I deploy the thing remotely, I get exception 
java.sql.SQLException: No suitable driver on line:


derbyconn = DriverManager.getConnection(dburl);

Note: the EmbeddedDriver jdbc class is most probably loaded fine, 
which is reinforced by the fact that if I delete derby.jar from 
classpath I get ClassNotFoundException.


Some people on the net suggested that this exception is being caused 
by wrong db url, but I can't see what's possibly wrong with my db url. 
I was under impression that this is correct dburl for embedded derby 
driver which should be in derby.jar.

Re: Java exception: '-1: java.lang.ArrayIndexOutOfBoundsException' error

[Kristian Waagan]

On 26.05.10 12:12, Shankar Devi wrote:

Hi,
  I have an intensive data-processing application which utilises Apache Derby
Version - 10.3.1.4.
My Customer is running this application on Windows 2003 Service Pack 2 and
has 1.6 GB derby database size.
Application has crashed after throwing this error.
   


Hi Devi,

Can you say which transaction isolation level the application is running 
with?

And I'm assuming this is a multi-threaded / -user application?
How easily is the bug reproducible?

Taking a guess from the stack trace it looks like you may be inserting 
data returned by a select.

Is this correct, and is the source data a BLOB or a CLOB?

You may have hit issue DERBY-3811 [1].


Regards,
--
Kristian

[1] https://issues.apache.org/jira/browse/DERBY-3811


Thanks,
Devi


May 7, 2010 3:50:47 PM java.sql.SQLException: Java exception: '-1:
java.lang.ArrayIndexOutOfBoundsException'.
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
Source)
at 
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown
Source)
at
com.netsation.configurator.server.DerbyStorage.saveObjectVersionTree(Unknown
Source)
at
com.netsation.configurator.server.DerbyStorage.saveObjectVersionTree(Unknown
Source)
at
com.netsation.configurator.server.DerbyStorage.saveObjectVersionTree(Unknown
Source)
at
com.netsation.configurator.server.DerbyStorage.saveObjectVersionTree(Unknown
Source)
at
com.netsation.configurator.server.DerbyStorage.saveObjectVersionTree(Unknown
Source)
at 
com.netsation.configurator.server.DerbyStorage.saveObjectVersion(Unknown
Source)
at com.netsation.configurator.server.Transaction.perform(Unknown Source)
at com.netsation.configurator.server.Transaction.transact(Unknown 
Source)
at 
com.netsation.configurator.server.Transaction.checkPointInternal(Unknown
Source)
at com.netsation.configurator.server.Transaction.checkPoint(Unknown 
Source)
at
com.netsation.configurator.common.StorageMechanism.checkPointTransaction(Unknown
Source)
at com.netsation.configurator.server.HardLimitCache.cleanObject(Unknown
Source)
at com.netsation.configurator.server.HardLimitCache.removeObject(Unknown
Source)
at com.netsation.configurator.server.HardLimitCache.removeTail(Unknown
Source)
at com.netsation.configurator.server.HardLimitCache.punt(Unknown Source)
at com.netsation.configurator.server.HardLimitCache.write(Unknown 
Source)
at 
com.netsation.configurator.common.StorageMechanism.writeToCache(Unknown
Source)
at com.netsation.configurator.common.StorageMechanism.loadObject(Unknown
Source)
at com.netsation.configurator.common.StorageMechanism.loadObject(Unknown
Source)
at com.netsation.configurator.server.LowLevelServices.loadObject(Unknown
Source)
at com.netsation.configurator.server.AccessService.loadAccess(Unknown
Source)
at com.netsation.configurator.server.AccessService.getAccess(Unknown
Source)
at com.netsation.configurator.server.CConfigurable.getAccess(Unknown
Source)
at com.netsation.configurator.server.CConfigurable.checkAccess(Unknown
Source)
at 
com.netsation.configurator.server.CConfigurable.representation(Unknown
Source)
at 
com.netsation.configurator.server.CConfigurable.representation(Unknown
Source)
at
com.netsation.configurator.server.NsConfigurationServer.propagateEventsToClients(Unknown
Source)
at
com.netsation.configurator.server.LowLevelServices.commitTransaction(Unknown
Source)
at
com.baynetworks.optivity.configurator.NetworkImportThread.postprocess(Unknown
Source)
at
com.baynetworks.optivity.configurator.NetworkImportThread.execute(Unknown
Source)
at com.netsation.util.ThreadedOperation.run(Unknown Source)
Caused by: java.sql.SQLException: Java exception: '-1:
java.lang.ArrayIndexOutOfBoundsException'.
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
Source)
... 41 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
at 

Re: Truncating CLOB error

[Kristian Waagan]

On 06.05.10 15:04, Mark Thornton wrote:

On 06/05/2010 13:52, Marco Ferretti wrote:

*maybe* you are affected by this
http://opensource.atlassian.com/projects/hibernate/browse/HHH-2614?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel



Very likely, especially as Mike Douglass is I believe one of the main
developers of Bedework. Would you expect Derby to be merely reporting a
warning for the truncation and that it would be Hibernate that is
promoting this to failure?


Hi Mark,

Derby is only allowed to truncate whitespace. If it needs to truncate 
non-whitespace characters to make the value fit, it will throw an 
exception (not a warning).



Regards,
--
Kristian



Thanks,
Mark Thornton

Re: A truncation error was encountered trying to shrink VARCHARTA 'XX-RESOLVE-XX' to length 255.

[Kristian Waagan]

)
 at
org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:293)
 at
org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:269)
 at
org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:261)
 at
org.codehaus.groovy.grails.web.mapping.filter.UrlMappingsFilter.doFilterInternal(UrlMappingsFilter.java:181)
 at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
 at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
 at
org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.obtainContent(GrailsPageFilter.java:171)
 at
org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.doFilter(GrailsPageFilter.java:110)
 at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
 at
org.codehaus.groovy.grails.web.servlet.filter.GrailsReloadServletFilter.doFilterInternal(GrailsReloadServletFilter.java:101)
 at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
 at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
 at
org.codehaus.groovy.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:65)
 at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
 at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
 at
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:96)
 at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
 at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
 at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
 at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
 at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
 at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
 at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
 at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
 at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
 at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
 at org.mortbay.jetty.Server.handle(Server.java:324)
 at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
 at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:879)
 at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:741)
 at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:213)
 at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
 at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
 at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
Caused by: org.codehaus.groovy.runtime.typehandling.GroovyCastException:
Cannot cast object 'org.hibernate.lob.serializableb...@1b5302d' with class
'o
rg.hibernate.lob.SerializableBlob' to class 'java.lang.Number'
 at CastMember.setProperty(CastMember.groovy)
 at CharacterController$_closure2.doCall(CharacterController:48)
 at CharacterController$_closure2.doCall(CharacterController)

Thanks in advance!!

Doug


Kristian Waagan-4 wrote:
   

On 26.04.10 17:45, OggieOne64 wrote:
 

Thanks for responding but the problem is that I am trying to insert an
image
(not text). The field is of type Blob.
Any other ideas? BTW: How can I verify what version of derby is running?
My
derby.jar is dated 8/10/2009
   

Hi Doug,

Can you post the stack trace from the exception?
Are you sure Hibernate is mapping the column to the right column type?

Regarding the Derby version, one way to figure it out is invoke the
command below with at least one of the Derby jars (i.e. derby.jar,
derbyclient.jar etc) in the classpath:
java -cpyourclasspath  org.apache.derby.tools.sysinfo
 or potentially
java -jar derbyrun.jar sysinfo

I don't remember exactly, but I think you can use JDBC meta data calls
as well. You can also look at some files inside one of the jars, for
instance the manifest file, and if you start ij the version is printed
(not the full version, i.e. only 10.5).


Regards,
--
Kristian

 

Thanks,
Doug



Knut Anders Hatlen wrote:
   

On 04/24/10 17:18, Doug McGavin wrote:
 

I am using the latest version of Derby with Hibernate and getting the
following error when trying to save an image (the exact

Re: Very bad disk space leak in Derby

[Kristian Waagan]

On 30.04.10 23:38, David Van Couvering wrote:

Hi, all.

I have a thread that runs the following query every 10 seconds:

SELECT d.ITEMSET_ID, f.FILTER_INDEX, FROM FILTERS f, PENDING_DELIVERIES
d WHERE f.ITEMSET_ID = d.ITEMSET_ID AND f.FILTER_INDEX = d.FILTER_INDEX
AND d.SENT = 0 AND d.SERVER_ID = 1

The FILTERS table has about 200 rows in it, and the PENDING_DELIVERIES
table has no rows.  So this query returns zero rows each time it is run.

My database directory is growing by about *1 MB *! every 30 seconds.

This is the *only* query running.  The number of rows in my tables never
changes.

Inspecting further, the number of dat files under seg0 is continually
growing.  This never stops - a QA person has seen their database
directory grow from 20MB to 5GB!


Hi David,

Can you confirm that the number of dat files in the seg0-directory are 
growing?
If that's the case, are you sure you don't have a query adding new 
tables and/or indexes to the database?
Do you see many tables if you fire up ij and issue 'show tables'? What 
about 'show indexes'?

(I think the values for an empty database are 22 and 72)


Regards,
--
Kristian



I tried compressing the table - no change.

I upgraded from 10.4 to 10.5, and the database initially was back to a
few MB, but then over time it started growing again.

This is *very* disconcerting - basically it makes Derby unusable in
production.  It is not an option for me to stop running this query.

I am hoping you have some suggestions, I am a bit at a loss.

Thanks!

David



--
David W. Van Couvering

http://www.linkedin.com/in/davidvc
http://davidvancouvering.blogspot.com
http://twitter.com/dcouvering