Re: Network server with different java (or derby) version between client and server

[Rick Hillegas]

Glad to hear that you got this to work. Some responses inline...
On 2/6/23 11:36 AM, fed wrote:
Hi,

Sorry for the late reply to this mail but I was able to create and test a
configuration and I have some doubts and questions.
I ask you if you can review my config that I describe in the mail.
I am sorry if it is a long email, I hope someone will read it :)

In my test environment I have the directory with databases in /tmp/1/ and
as derby installation directory /tmp/2/ so I have
/tmp/2/db-derby-10.15.2.0-bin/lib.

I put my databases in /tmp/1 so I have /tmp/1/db1 , /tmp/1/db2 etc. and I
created in the same dir a new database named credentials where I enabled
native authentication that will be used by the network server for shutdown
with user/password.

I created /tmp/1/derby.properties file with content:

derby.authentication.provider=NATIVE:credentials

I edited the server template that comes from the templates dir in the demo
dir (I paste it at the end of the email) and I put it
/tmp/2/db-derby-10.15.2.0-bin/lib.

To run the server I use, from /tmp/2/db-derby-10.15.2.0-bin/lib :

java -Djava.security.policy=serverTemplate.policy
-Dderby.system.home=/tmp/1  -jar derbyrun.jar server start

Every database in /tmp/1 has native authentication enabled, so every
database has its own user/password different for each one.

It seems to work: I can connect to the various databases and to shutdown
the server I have to use

java -jar derbyrun.jar server shutdown -user someuser -password somepassword

where someuser and somepassword are the ones that I defined in the
credentials database.

Is my test environment/configuration correct ?
This sounds good. Since you're able to connect to databases with 
credentials and also shutdown the server, it sounds like you've 
configured your installation correctly.

I have other questions:

1 - it’s not clear to me how credentials db used by the server affects the
creation of new databases. I see that to create a database I have to use
the same credentials I saved in the credentials database, but if I have a
database just created, so not new, with its owner defined this will not
affect this database, right? Also It’s not clear if I have to use
derby.authentication.provider=NATIVE:credentials or
derby.authentication.provider=NATIVE:credentials:LOCAL for server
credentials in /tmp/1/derby.properties.

derby.authentication.provider=NATIVE:credentials

sounds like the right choice to me. It designates the "credentials" 
database as the system-wide definition of who is allowed to create a 
database connection. All databases defer to the "credentials" database 
for authentication.
This can be ok, what I would like is to disable or make passwords protect
the creation of databases on the server but in most cases I will use
databases that I created before I put them on the server, with their
user/password so if this is how it works for me is ok.
Unlike many other databases, Derby doesn't have a special permission to 
control who can create databases. Anyone with valid credentials can 
create a database by supplying their credentials and specifying 
"create=true" on the connection URL.

2 - Does the shutdown of the server imply the shutdown of every database ?
Is it enough ? I don’t have to call shutdown to every database individually
I suppose.
Shutting down the server will gracefully shutdown all open databases.

Thanks for the help, I copy my server policy file where I just substituted
${derby.install.url} with file:///tmp/2/db-derby-10.15.2.0-bin/lib/.

-Fed

// Licensed to the Apache Software Foundation (ASF) under one or more
// contributor license agreements.  See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// The ASF licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License.  You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

grant codeBase "file:///tmp/2/db-derby-10.15.2.0-bin/lib/derbyshared.jar"
{
   // Needed to determine whether the JVM was booted with a module path.
   permission java.util.PropertyPermission "jdk.module.path", "read";

   // Need in order to read message localizations from other derby jars
   // when running with a module path.
   permission java.io.FilePermission "${derby.install.path}${/}-", "read";

   // Gives permission for JMX to be used against Derby
   // but only if JMX authentication is not being used.
   // In that case the application would need to create
   // a whole set of fine-grained permissions to allow specific
   // users access to MBeans and actions they perform.
   permission org.apache.derby.shared.common.security.SystemPermission
"engine", "monitor";
   permission org.apache.derby.shared.common.security.SystemPermission
"server", "monitor";

};

grant codeBase "file:///tmp/2/db-derby-10.15.2.0-bin/lib/derby.jar"
{
   // These permissions are needed for everyday, embedded Derby usage.
   permission java.lang.RuntimePermission "createClassLoader";
   permission java.util.PropertyPermission "derby.*", "read";
   permission java.util.PropertyPermission "user.dir", "read";
   permission org.apache.derby.shared.common.security.SystemPermission
"engine", "usederbyinternals";
   permission java.io.FilePermission "${derby.system.home}","read,write";
   permission java.io.FilePermission "${derby.system.home}${/}-",
"read,write,delete";
   permission java.io.FilePermission
"${derby.system.home}${/}derby.properties", "read";
   permission java.io.FilePermission
"${derby.system.home}${/}derby.log", "read,write,delete";

   // Properties needed to determine if the VM is 32 or 64 bit.
   permission java.util.PropertyPermission "sun.arch.data.model", "read";
   permission java.util.PropertyPermission "os.arch", "read";

   // Gives permission for JMX to be used against Derby
   // but only if JMX authentication is not being used.
   // In that case the application would need to create
   // a whole set of fine-grained permissions to allow specific
   // users access to MBeans and actions they perform.
   permission org.apache.derby.shared.common.security.SystemPermission
"jmx", "control";
   permission org.apache.derby.shared.common.security.SystemPermission
"engine", "monitor";
   permission org.apache.derby.shared.common.security.SystemPermission
"server", "monitor";

   // Permissions needed for JMX based management and monitoring.
   permission javax.management.MBeanServerPermission "createMBeanServer";

   // Allows access to Derby's built-in MBeans, within the domain
   // org.apache.derby.  Derby must be allowed to register and unregister these
   // MBeans.  To fine tune this permission, see the javadoc of
   // javax.management.MBeanPermission or the JMX Instrumentation and
Agent Specification.
   permission javax.management.MBeanPermission
"org.apache.derby.*#[org.apache.derby:*]",
"registerMBean,unregisterMBean";

   // Trusts Derby code to be a source of MBeans and to register these
in the MBean server.
   permission javax.management.MBeanTrustPermission "register";

   // Optional permission needed for printing classpath information to 
derby.log.
   permission java.lang.RuntimePermission "getProtectionDomain";

   // The following permission must be granted for
Connection.abort(Executor) to work.
   // Note that this permission must also be granted to outer
(application) code domains.
   permission java.sql.SQLPermission "callAbort";

   // Needed by FileUtil#limitAccessToOwner.
   permission java.lang.RuntimePermission "accessUserInformation";
   permission java.lang.RuntimePermission "getFileStoreAttributes";

   // Needed to create a temp file in order to open a database in a jar file.
   // permission java.io.FilePermission "${java.io.tmpdir}${/}-",
"read,write,delete"

   // Customize the following permission in order to backup and restore
   // Derby databases to/from a secure branch of your file system,
   // preferably one owned by the database owner or the user who booted the JVM:
   // permission java.io.FilePermission "/Users/me/backups/-", "read,write"

   // Customize the following permission in order to export and import
   // tables to/from a secure branch of your file system,
   // preferably one owned by the database owner or the user who booted the JVM:
   // permission java.io.FilePermission "/Users/me/imports/-", "read,write"

   // Customize the following permission in order to load
   // jar files which contain user-written types, aggregates,
functions, and procedures.
   // Those jar files should live in a secure branch of your file system,
   // preferably one owned by the database owner or the user who booted the JVM:
   // permission java.io.FilePermission "/Users/me/myJars/-", "read"

   // This permission lets a DBA reload the policy file while the server
   // is still running. The policy file is reloaded by invoking
   // the SYSCS_UTIL.SYSCS_RELOAD_SECURITY_POLICY() system procedure:
   // permission java.security.SecurityPermission "getPolicy"

   // This permission is needed to call DriverManager.deregisterDriver()
   // and unload the Derby classes:
   // permission java.sql.SQLPermission "deregisterDriver"

};

grant codeBase "file:///tmp/2/db-derby-10.15.2.0-bin/lib/derbytools.jar"
{
   // Access all properties using System.getProperties -
   // ij enumerates the properties in order to open connections
   // for any property set in ij.connection.* and set protocols
   // for any property in ij.protocol.*
   permission java.util.PropertyPermission "*", "read,write";

   // Needed by sysinfo. The file permission is needed to check the existence
   // of jars on the classpath. You can limit this permission to just
the locations
   // which hold your jar files.
   permission java.util.PropertyPermission "user.*", "read";
   permission java.util.PropertyPermission "java.home", "read";
   permission java.util.PropertyPermission "java.class.path", "read";
   permission java.util.PropertyPermission "java.runtime.version", "read";
   permission java.util.PropertyPermission "java.fullversion", "read";
   permission java.lang.RuntimePermission "getProtectionDomain";
   permission org.apache.derby.shared.common.security.SystemPermission
"engine", "usederbyinternals";
   permission java.io.FilePermission "<<ALL FILES>>", "read";

};

grant codeBase "file:///tmp/2/db-derby-10.15.2.0-bin/lib/derbynet.jar"
{
   // Needed by FileUtil#limitAccessToOwner.
   permission java.lang.RuntimePermission "accessUserInformation";
   permission java.lang.RuntimePermission "getFileStoreAttributes";
   permission java.util.PropertyPermission
"derby.__serverStartedFromCmdLine", "read,write";

   // Needed for NetworkServerMBean access.
   permission org.apache.derby.shared.common.security.SystemPermission
"server", "control,monitor";
   permission org.apache.derby.shared.common.security.SystemPermission
"engine", "usederbyinternals";

   // Accept connections from any host. Derby is listening to the host interface
   // specified via the -h command line option to "NetworkServerControl start",
   // via the address parameter to the 
org.apache.derby.drda.NetworkServerControl
   // constructor in the API, or via the property derby.drda.host.
   // The default is localhost.  You may want to restrict allowed hosts,
   // e.g. to hosts in a specific subdomain like "*.example.com".
   permission java.net.SocketPermission "*", "accept";

   // Allow the server to listen to the socket on the port specified with the
   // -p option to "NetworkServerControl start" on the command line, or with
   // the portNumber parameter to the NetworkServerControl constructor in the
   // API, or with the property derby.drda.portNumber. The default is 1527.
   permission java.net.SocketPermission
"localhost:${derby.security.port}", "listen";

   // Needed for server tracing.
   permission java.io.FilePermission
"${derby.drda.traceDirectory}${/}-", "read,write,delete";

};


On Thu, 1 Dec 2022 at 21:13, Rick Hillegas <rick.hille...@gmail.com> wrote:

Some responses inline...

On 12/1/22 4:15 AM, fed wrote:

Hi,

As you suggested, the issue is the security manager, to be honest I am used
to java security manager, I read and tested a bit to understand how it
works.
My initial setup was server service in a dir, let’s say /dir1 and database
in another dir let’s say /dir2 so the dirs are distinct to each other.

Considering version 10.15.2.0 running the server without security manager
works:

java -jar derbyrun.jar server -noSecurityManager start

but reading from documentation I know it is not recommended and advisable
so I want to use the security manager.

So then I put my db dir inside the lib dir where derbyrun.jar and other
jars are, so something like lib/db/mydb and running it with
java -jar derbyrun.jar server start
and I can access the db even using relative path on jdbc url, db/mydb.

I am not used to derby in server mode, I read the documentation but sorry
it’s not all clear to me how to use it.
So from my test I suppose the recommended setup is to create a db dir
inside lib and put all the dbs inside it ? lib/db/db1, lib/db/db2 etc ?
right?

Typically, people separate code-bearing directories from data-bearing
directories. This makes it easier to upgrade to a new version of Derby
since the database is not located with the old version of Derby code.
Something like the following:

/Users/me/derbyInstallation/

/Users/me/derbyInstallation/derbyVersions/10.15.2.0

/Users/me/derbyInstallation/derbyHome

/Users/me/derbyInstallation/derbyHome/db1

/Users/me/derbyInstallation/derbyHome/db2


Just as note I tested again with 10.12.1.1 and I found that forcing the
security policy with the server template one (I suppose the default
behaviour is changed with the newer version) it behaves the same like
10.15.2.0, I used:
java
-Djava.security.policy=/tmp/db-derby-10.12.1.1-bin/demo/templates/server.policy
-jar derbyrun server start

Note that the supplied policies are just templates. You need to edit them
in order to use them. See
https://db.apache.org/derby/docs/10.15/security/csecjavasecurity.html
That is, at a minimum, you need to copy the template server policy (for
10.15.2.0 it's demo/templates/serverTemplate.policy) to some other location
and edit lines like the following...

grant codeBase "${derby.install.url}derbyshared.jar"


...replacing the ${derby.install.url} symbol with the url handle of your
Derby code library (file:///Users/me/derbyInstallation/10.15.2.0/) so
that the lines look like this:

grant codeBase "file:///Users/me/derbyInstallation/10.15.2.0/derbyshared.jar"

Hope this helps,

-Rick

Thanks for the help.

On Fri, 25 Nov 2022 at 21:02, Rick Hillegas <rick.hille...@gmail.com> 
<rick.hille...@gmail.com> wrote:


Check that your 10.15 classpath is correct. You need a couple more jar
files compared to previous releases. Your 10.15 server classpath must
contain the following jars:

    derby.jar
    derbyshared.jar
    derbytools.jar
    derbynet.jar

See
https://db.apache.org/derby/docs/10.15/adminguide/tadminappschangingyourclasspath.html
and
https://db.apache.org/derby/docs/10.15/publishedapi/org.apache.derby.server/module-summary.html


On 11/25/22 9:51 AM, Rick Hillegas wrote:

This indicates that the server is running with a Java SecurityManager
and that the policy file does not grant read permission on that
file--and probably all files in the database directory.

On 11/25/22 12:30 AM, fed wrote:

Hi,

testing with 10.15.2.0 from derby.log, server side, it complains about a
read permission on service.properties, some part of the file:

java.sql.SQLException: Impossibile avviare il database
'/home/user/db/' con
il caricatore di classi
jdk.internal.loader.ClassLoaders$AppClassLoader@277050dc. Per i
dettagli,
vedere l'eccezione successiva.
...
Caused by: java.security.AccessControlException: access denied
("java.io.FilePermission" "/home/user/db/service.properties" "read")
...
ERROR XBM0C: Privilegio mancante per l'operazione 'exists' sul file
'service.properties': access denied ("java.io.FilePermission"
"/home/user/db/service.properties" "read")


There are several errors like these ones but I have read permission
on this
file.
The user that starts the server is the same that owns the file, the
permissions on the file are 664.

As I said, same setup but using 10.12.1.1 for the server, I have no
problems.

Best Regards
-fed

On Thu, 24 Nov 2022 at 19:52, Rick Hillegas <rick.hille...@gmail.com> 
<rick.hille...@gmail.com>
wrote:


The SQLState indicates that the server was not able to boot the
database. Look in the server-side derby.log to see if there is a
detailed error message describing why the boot failed.

On 11/23/22 4:42 PM, fed wrote:

Hi,

Sorry for the late answer but I lost your reply.

Two tests:

I have a database updated to version 10.12.1.1, the server is running

with

the 10.12.1.1 too and the client is using 10.12.1.1 too, the
connection

is

OK, I can use this setup.

But another test:
still the same database updated to version 10.12.1.1, the server is

running

10.15.2.0 so a newer version and the client is using 10.12.1.1: I

have

problems in this case the client can't connect to the database with
this
error:

Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error:
ERRORCODE: 40000, SQLSTATE: XJ040, SQLERRMC: Impossibile avviare il
database '/home/user/some_db_path/' con il caricatore di classi
jdk.internal.loader.ClassLoaders$AppClassLoader@277050dc. Per i

dettagli,

vedere l'eccezione successiva.::SQLSTATE: XBM0C

Thanks for the help



On Sun, 13 Nov 2022 at 15:26, Bryan Pendleton <

bpendleton.de...@gmail.com>

wrote:


I'm not aware of client-server version incompatibilities. Have you
done any experiments with different versions?

thanks,

bryan

On Thu, Nov 10, 2022 at 4:16 AM fed <fury...@gmail.com> <fury...@gmail.com> 
wrote:

Hi,

using derby with network server setup is there any problem if the

server

and the client are running on different java versions?

Still on this, considering the database created/updated with the
apache

derby version that the client uses, is there any problem if the
server

will

use a newer version of apache derby?

Thanks for the help

-fed