Re: OpenSSL SSL_CONF_cmd API

2017-08-16 Thread Rainer Jung

Am 16.08.2017 um 14:45 schrieb Rainer Jung:

Am 13.08.2017 um 19:37 schrieb Mark Thomas:

On 13/08/17 16:42, Rainer Jung wrote:

Hi,

OpenSSl has an API named SSL_CONF_cmd. Te API allows application using
OpenSSL to no longer implement an application specific configuration
option per OpenSSL config feature the app wants to support, but instead
use a more generic approach. The API can be seen here (it was added in
1.0.2, but also exists in 1.1.0):

https://www.openssl.org/docs/man1.0.2/ssl/SSL_CONF_cmd.html

mod_ssl in httpd already supports that API:

http://httpd.apache.org/docs/2.4/en/mod/mod_ssl.html#sslopensslconfcmd

The API can be fed with pairs of command names and values. If we would
like to support this, we would have to find an appropriate approach of
supporting name/value pairs in our config. I'd say server.xml (and every
xml based file) isn't appropriate.

Is there any interest in supporting SSL_CONF_cmd?

If so, should we

a) create a new file, e.g. conf/openssl.properties
or
b) pack new properties into catalina.properties, probably with a common
prefix "openssl.conf.cmd."?

And if catalina.properties, should we add the properties also to java
system properties or filter them?

Or maybe the other way round: extract all system properties named
openssl.conf.cmd.* and use them?

If we would want to support different setting per connector or
SSLHostConfig, we would instead need a properties file per connector or
SSLHostConfig, so probably an xml attribute opensslConfCmdFile="..." and
read the properties form there.

Any opinion?


I like the idea of not having to replicate openssl setters and gettes in
Java and C.

I think the configuration needs to be per SSLHostConfig.

I was thinking of something along the line of IntrosepctionUtils that
was passed the attribute name and value from the XML and then called the
appropriate API. If the attributes had a common prefix (openssl.cmd...)
then ID'ing the name/value pairs should be fairly easy.

Haven't thought about how this might integrate with the current code.


I think implementation of handling the specific SSL_CONF_cmd attributes
would be easier (and cleaner), if we introduce another SSLHostConfig sub
element (analogous to ), e.g. named , and
each attribute there gets set by calling a generic method using key and
value as args instead of a key-specific attribute setter. That's most
likely easier to implement with digester and produces less config magic
plus we no longer need such an attribute name prefix. It kind of is
implicit in the new sub element.

I'll try to do a prototype later and if it works already include the
needed native calls in the tcnative, so that we'll have something in
1.2.13 that we can build upon.

Since at config parsing time we do not already have the SSL context
available that needs to be passed when calling SSL_CONF_cmd(), we would
need to store the list of key/values during config parse time and then
later when creating the ssl context we need to apply them. That is quite
similar to what happens in mod_ssl. Storing the list would be easier in
the Java world and keep the native part simpler. In addition to the
SSL_CONF_cmd() calls when actually setting up the ssl context, we could
do some checks for the key/value pairs already during config parse time
using SSL_CONF_cmd_value_type() and code similar to mod_ssl
(ssl_cmd_SSLOpenSSLConfCmd).


There seems to be a conceptual problem. The order of attributes seems to 
not be well-defined, but the order of SSL_CONF_cmd executions can be 
relevant, so we should aim at getting the commands in the order they 
have been put into the config file. I guess we need the more ugly config 
style (beneath SSLHostConfig):




...



since elements should already get processed in an ordered way.

Regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



buildbot success in on tomcat-7-trunk

2017-08-16 Thread buildbot
The Buildbot has detected a restored build on builder tomcat-7-trunk while 
building . Full details are available at:
https://ci.apache.org/builders/tomcat-7-trunk/builds/845

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: silvanus_ubuntu

Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-7-commit' 
triggered this build
Build Source Stamp: [branch tomcat/tc7.0.x/trunk] 1805229
Blamelist: violetagg

Build succeeded!

Sincerely,
 -The Buildbot




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r1805229 - /tomcat/tc7.0.x/trunk/webapps/docs/changelog.xml

2017-08-16 Thread violetagg
Author: violetagg
Date: Wed Aug 16 18:27:02 2017
New Revision: 1805229

URL: http://svn.apache.org/viewvc?rev=1805229=rev
Log:
Update the release date for 7.0.81

Modified:
tomcat/tc7.0.x/trunk/webapps/docs/changelog.xml

Modified: tomcat/tc7.0.x/trunk/webapps/docs/changelog.xml
URL: 
http://svn.apache.org/viewvc/tomcat/tc7.0.x/trunk/webapps/docs/changelog.xml?rev=1805229=1805228=1805229=diff
==
--- tomcat/tc7.0.x/trunk/webapps/docs/changelog.xml (original)
+++ tomcat/tc7.0.x/trunk/webapps/docs/changelog.xml Wed Aug 16 18:27:02 2017
@@ -59,7 +59,7 @@
 -->
 
 
-
+
   
 
   



-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



svn commit: r21191 - /dev/tomcat/tomcat-7/v7.0.81/ /release/tomcat/tomcat-7/v7.0.81/

2017-08-16 Thread violetagg
Author: violetagg
Date: Wed Aug 16 18:22:18 2017
New Revision: 21191

Log:
Release 7.0.81

Added:
release/tomcat/tomcat-7/v7.0.81/
  - copied from r21190, dev/tomcat/tomcat-7/v7.0.81/
Removed:
dev/tomcat/tomcat-7/v7.0.81/


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: [VOTE] Release Apache Tomcat 8.0.46

2017-08-16 Thread Violeta Georgieva
Hi,

2017-08-10 13:55 GMT+03:00 Violeta Georgieva :
>
> The proposed Apache Tomcat 8.0.46 release is now available for voting.
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.0.46/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1150/
> The svn tag is:
> http://svn.apache.org/repos/asf/tomcat/tc8.0.x/tags/TOMCAT_8_0_46/
>
> The proposed 8.0.46 release is:
> [ ] Broken - do not release
> [ ] Stable - go ahead and release as 8.0.46

We need more votes here.

Thanks,
Violeta


Re: [RESULT][VOTE] Release Apache Tomcat 7.0.81

2017-08-16 Thread Violeta Georgieva
Hi,

2017-08-11 14:14 GMT+03:00 Violeta Georgieva :
>
> The proposed Apache Tomcat 7.0.81 release is now available for voting.
>
> It can be obtained from:
> https://dist.apache.org/repos/dist/dev/tomcat/tomcat-7/v7.0.81/
> The Maven staging repo is:
> https://repository.apache.org/content/repositories/orgapachetomcat-1151/
> The svn tag is:
> http://svn.apache.org/repos/asf/tomcat/tc7.0.x/tags/TOMCAT_7_0_81/
>
> The proposed 7.0.81 release is:
> [ ] Broken - do not release
> [ ] Stable - go ahead and release as 7.0.81 Stable

+1 (binding):huxing, violetagg, mgrigorov

No other voters were cast.

The vote has passed.

I'll do the release shortly and announce it once the mirrors catch up.

Regards,
Violeta


[Bug 61425] all idle connections become ' in transaction' when the 'testWhileIdle' is set to 'true' and 'defaultAutoCommit' is set to 'false'

2017-08-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61425

--- Comment #4 from WangZheng  ---
admin=# select current_query from pg_stat_activity where usename='admin';
 current_query 
---
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
  in transaction
(15 rows)

The above is queried the database.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 61425] all idle connections become ' in transaction' when the 'testWhileIdle' is set to 'true' and 'defaultAutoCommit' is set to 'false'

2017-08-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61425

--- Comment #3 from WangZheng  ---
Created attachment 35233
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=35233=edit
use this patch instead

Sorry, there are some mistakes in the previous patch, please use one instead.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 61425] all idle connections become ' in transaction' when the 'testWhileIdle' is set to 'true' and 'defaultAutoCommit' is set to 'false'

2017-08-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61425

--- Comment #2 from WangZheng  ---
Comment on attachment 35232
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=35232
Attaching a simple patch to commit/rollback the connection.

diff --git a/org/apache/tomcat/jdbc/pool/PooledConnection.java
b/org/apache/tomcat/jdbc/pool/PooledConnection.java
index c833209..4615abf 100644
--- a/org/apache/tomcat/jdbc/pool/PooledConnection.java
+++ b/org/apache/tomcat/jdbc/pool/PooledConnection.java
@@ -541,6 +541,22 @@ public class PooledConnection {
 }
 if (stmt!=null)
 try { stmt.close();} catch (Exception ignore2){/*NOOP*/}
+
+try {
+if(!connection.getAutoCommit()) {
+connection.rollback();
+}
+} catch (SQLException e) {
+// do nothing
+}
+} finally {
+try {
+if(!connection.getAutoCommit()) {
+connection.commit();
+}
+} catch (SQLException e) {
+// do nothing
+}
 }
 return false;
 } //validate

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 61425] all idle connections become ' in transaction' when the 'testWhileIdle' is set to 'true' and 'defaultAutoCommit' is set to 'false'

2017-08-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61425

--- Comment #1 from WangZheng  ---
Created attachment 35232
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=35232=edit
Attaching a simple patch to commit/rollback the connection.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 61425] New: all idle connections become ' in transaction' when the 'testWhileIdle' is set to 'true' and 'defaultAutoCommit' is set to 'false'

2017-08-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61425

Bug ID: 61425
   Summary: all idle connections become ' in transaction'
when the 'testWhileIdle' is set to 'true' and
'defaultAutoCommit' is set to 'false'
   Product: Tomcat Modules
   Version: unspecified
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P2
 Component: jdbc-pool
  Assignee: dev@tomcat.apache.org
  Reporter: rainbow...@139.com
  Target Milestone: ---

In my application, I use spring boot framework, and I choose "Tomcat JDBC Pool"
as my connection pool, what I have configured is as follows.

spring.datasource.type=org.apache.tomcat.jdbc.pool.DataSource
spring.datasource.url=jdbc:postgresql://localhost:5432/test
spring.datasource.tomcat.driverClassName=org.postgresql.Driver
spring.datasource.tomcat.username=admin
spring.datasource.tomcat.password=admin
spring.datasource.tomcat.initialSize=15
spring.datasource.tomcat.min-idle=15
spring.datasource.tomcat.maxActive=30
spring.datasource.tomcat.max-idle=30
spring.datasource.tomcat.maxWait=30
spring.datasource.tomcat.timeBetweenEvictionRunsMillis=3
spring.datasource.tomcat.minEvictableIdleTimeMillis=6
spring.datasource.tomcat.removeAbandoned=true
spring.datasource.tomcat.removeAbandonedTimeout=900
spring.datasource.tomcat.logAbandoned=true

spring.datasource.tomcat.testWhileIdle=true
spring.datasource.tomcat.validationQuery=SELECT 1
spring.datasource.tomcat.validationInterval=6
spring.datasource.tomcat.default-auto-commit=false

Then I run my application, 60 seconds later, all the database connections
become into " in transaction".

Why does this happen? I look into the "PooledConnection.java" source code, and
I get the reason.

In the above configuration, I set the "testWhileIdle" to "true", so every
connection in the pool will be tested to see if it is idle, using the specified
SQL "SELECT 1".
After executing the SQL, the connection does not call commit() or rollback().
We know that if the connection is auto committed, this is OK.
But because I set the "defaultAutoCommit" to "false", so the connection will
not be committed automatically, and so the connection will always stay in the
status of " in transaction", and I think this is incorrect.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: OpenSSL SSL_CONF_cmd API

2017-08-16 Thread Rainer Jung

Am 13.08.2017 um 19:37 schrieb Mark Thomas:

On 13/08/17 16:42, Rainer Jung wrote:

Hi,

OpenSSl has an API named SSL_CONF_cmd. Te API allows application using
OpenSSL to no longer implement an application specific configuration
option per OpenSSL config feature the app wants to support, but instead
use a more generic approach. The API can be seen here (it was added in
1.0.2, but also exists in 1.1.0):

https://www.openssl.org/docs/man1.0.2/ssl/SSL_CONF_cmd.html

mod_ssl in httpd already supports that API:

http://httpd.apache.org/docs/2.4/en/mod/mod_ssl.html#sslopensslconfcmd

The API can be fed with pairs of command names and values. If we would
like to support this, we would have to find an appropriate approach of
supporting name/value pairs in our config. I'd say server.xml (and every
xml based file) isn't appropriate.

Is there any interest in supporting SSL_CONF_cmd?

If so, should we

a) create a new file, e.g. conf/openssl.properties
or
b) pack new properties into catalina.properties, probably with a common
prefix "openssl.conf.cmd."?

And if catalina.properties, should we add the properties also to java
system properties or filter them?

Or maybe the other way round: extract all system properties named
openssl.conf.cmd.* and use them?

If we would want to support different setting per connector or
SSLHostConfig, we would instead need a properties file per connector or
SSLHostConfig, so probably an xml attribute opensslConfCmdFile="..." and
read the properties form there.

Any opinion?


I like the idea of not having to replicate openssl setters and gettes in
Java and C.

I think the configuration needs to be per SSLHostConfig.

I was thinking of something along the line of IntrosepctionUtils that
was passed the attribute name and value from the XML and then called the
appropriate API. If the attributes had a common prefix (openssl.cmd...)
then ID'ing the name/value pairs should be fairly easy.

Haven't thought about how this might integrate with the current code.


I think implementation of handling the specific SSL_CONF_cmd attributes 
would be easier (and cleaner), if we introduce another SSLHostConfig sub 
element (analogous to ), e.g. named , and 
each attribute there gets set by calling a generic method using key and 
value as args instead of a key-specific attribute setter. That's most 
likely easier to implement with digester and produces less config magic 
plus we no longer need such an attribute name prefix. It kind of is 
implicit in the new sub element.


I'll try to do a prototype later and if it works already include the 
needed native calls in the tcnative, so that we'll have something in 
1.2.13 that we can build upon.


Since at config parsing time we do not already have the SSL context 
available that needs to be passed when calling SSL_CONF_cmd(), we would 
need to store the list of key/values during config parse time and then 
later when creating the ssl context we need to apply them. That is quite 
similar to what happens in mod_ssl. Storing the list would be easier in 
the Java world and keep the native part simpler. In addition to the 
SSL_CONF_cmd() calls when actually setting up the ssl context, we could 
do some checks for the key/value pairs already during config parse time 
using SSL_CONF_cmd_value_type() and code similar to mod_ssl 
(ssl_cmd_SSLOpenSSLConfCmd).


Regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 61424] New: Obtaining a StackOverflowError when running Tomcat 8.5 or 9 with SecurityManager, a javax.management.remote.JMXPrincipal entry is present in catalina.policy file and Subject.doAs meth

2017-08-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61424

Bug ID: 61424
   Summary: Obtaining a StackOverflowError when running Tomcat 8.5
or 9 with SecurityManager, a
javax.management.remote.JMXPrincipal entry is present
in catalina.policy file and Subject.doAs method is
called.
   Product: Tomcat 8
   Version: 8.5.20
  Hardware: PC
Status: NEW
  Severity: normal
  Priority: P2
 Component: Catalina
  Assignee: dev@tomcat.apache.org
  Reporter: robert.gher...@museglobal.ro
  Target Milestone: 

Created attachment 35231
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=35231=edit
Project used to reproduce the StackOverflowError

When run Tomcat 8.5.20 with SecurityManager and catalina.policy contains an
javax.management.remote.JMXPrincipal entry and Subject.doAs method is called,
then a StackOverflowError is thrown.
Same error is reproducible in Tomcat 9.0.0.M26, but is not present in Tomcat
8.0.45.

The test was made using JDK 1.8.0_144.

In order to reproduce this error, we build a short example(see JMXSubject.war
in attachment) containing a simple servlet with the following source code:

package servlet;

import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.security.Principal;
import java.security.PrivilegedExceptionAction;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;

import javax.security.auth.Subject;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

@WebServlet("/")
public class MyServlet extends HttpServlet {

  private static final long serialVersionUID = -1647039991464261998L;

  @Override
  protected void doGet(HttpServletRequest reqest, HttpServletResponse response)
throws ServletException, IOException {
Object doAsResult = null;
Set principals = new HashSet<>();
principals.add(new Principal() {
  @Override
  public String getName() {
return "myName";
  }
});
Subject subject = new Subject(false, principals, Collections.EMPTY_SET,
Collections.EMPTY_SET);
try {
  doAsResult = Subject.doAs(subject, new
PrivilegedExceptionAction() {
public Object run() throws IllegalAccessException,
InvocationTargetException {
  return System.currentTimeMillis();
}
  });
} catch (Exception e) {
  e.printStackTrace();
}

response.getWriter().println("CurrentTimeMillis: " + doAsResult);
  }
}

Step to reproduce:
1) Deploy the provided JMXSubject.war web project (which also includes in the
archive the source code) in ${catalina.home}/webapps folder.

2) Add in ${catalina.home}/conf/catalina.policy file the following lines:

grant codeBase "file:/-" {
  permission java.security.AllPermission;
};

grant principal javax.management.remote.JMXPrincipal "jmx" {
  permission java.security.AllPermission;
};

3) Start server with SecurityManager:
catalina.bat run -security

4) Access the following page: http://localhost:8080/JMXSubject

Now the following exception is thrown in tomcat 8.5.20:

javax.servlet.ServletException
   
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:337)
   
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
java.security.AccessController.doPrivileged(Native Method)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
java.security.AccessController.doPrivileged(Native Method)
javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
   
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
   
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
Root Cause

java.lang.StackOverflowError
java.security.AccessController.doPrivileged(Native Method)
java.io.FilePermission.init(FilePermission.java:203)
java.io.FilePermission.(FilePermission.java:277)
java.lang.SecurityManager.checkRead(SecurityManager.java:888)
java.io.File.isDirectory(File.java:844)
sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:269)
   
sun.security.provider.PolicyFile.canonicalizeCodebase(PolicyFile.java:1735)

[Bug 61422] New: Feature requests for tc-native based on forked netty-tcnative

2017-08-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=61422

Bug ID: 61422
   Summary: Feature requests for tc-native based on forked
netty-tcnative
   Product: Tomcat Native
   Version: unspecified
  Hardware: PC
Status: NEW
  Severity: normal
  Priority: P2
 Component: Library
  Assignee: dev@tomcat.apache.org
  Reporter: matej.spil...@gmail.com
  Target Milestone: ---

There is a forked project from tcnative that is going to be used in netty.
https://github.com/netty/netty-tcnative

There are couple of very nice features, that I would like to have it in tomcat
version.

- mavenization of the project (I can include it as dependency -> very nice for
microservices where tomcat is embedded -> we use spring boot)
- additional build with dynamic linking for using latest upstream security
updates for openssl
- removal of APR in favour of c++11 / c++14 features
(https://github.com/netty/netty-tcnative/pull/255)
- additional support for boringssl, libressl
- visual studio express build support
- they claim better openssl support (i don't know what that really means)
- posibility to have one release for netty and tomcat (wishful thinking)

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org