Re: SSL on Tomcat7 on AWS not connecting

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

George,

On 11/17/16 4:48 PM, George Chanady wrote:
> Chris,
> 
> I tried curl with the -tls1 switch and received the same error.
> 
> [ec2-user@ip-172-31-52-159 bin]$ curl -vk
> https://bageoconsultants.com:8443 -tls1 * Rebuilt URL to:
> https://bageoconsultants.com:8443/ *   Trying 52.54.85.95... *
> Connected to bageoconsultants.com (52.54.85.95) port 8443 (#0) *
> Initializing NSS with certpath: sql:/etc/pki/nssdb * NSS error
> -12286 (SSL_ERROR_NO_CYPHER_OVERLAP) * Cannot communicate securely
> with peer: no common encryption algorithm(s). * Closing connection
> 0 curl: (35) Cannot communicate securely with peer: no common
> encryption algorithm(s).
> 
> I also tried with openssl s_client which was the last few lines of
> my of the original attachment. Also a no go,
> 
> [ec2-user@ip-172-31-34-217 conf]$ openssl s_client -connect
> bageoconsultants.com:8443 CONNECTED(0003) 
> 140427891013472:error:14077410:SSL
> routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake
> failure:s23_clnt.c:770: --- no peer certificate available --- No
> client certificate CA names sent --- SSL handshake has read 7 bytes
> and written 249 bytes --- New, (NONE), Cipher is (NONE) Secure
> Renegotiation IS NOT supported Compression: NONE Expansion: NONE 
> ---

s_client will also need the same switch, except it's like this for
s_client:

$ openssl s_client -tls1 -connect bageoconsultants.com:8443

That's not a publicly-accessible port or I would be able to check it
myself.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYL10YAAoJEBzwKT+lPKRYwwsQAKmZbWizrj4nkcF9pBCtDcV/
SngmsgtyTKgljJFGBr2ii3StHux06pCrWwsVN+/nIS9KFtZ4bgRWJ00RssqFbJgP
gaEdQf0av6wr04S7J0t6ub5ZsOpsR07YczCLCO1oe7ri6aOGrilulYTGx/e1K4H/
ry3r8QmyXaqoA6Y8NTA7mnibGUDH+AfB10acGf65lKaXzWsKm2PFdZD4ICwfsUQt
G5H4ydtIdOg9t4+FQpwWrOAjopnAysvtMJ6GmUKSXuoWTGjucUktIDfVB0KknczL
0JNn7O3HZ8H+mPEJwuckMjcLfRHVUE7GsZoHsyR1XJve6kPj2dJGmgAb26ygFb9X
ZCY8/jO99Zo64NSkoEWyHOgn1WFBshvrgHiW2J+O8FU72JCK8AQU62J4KlGWA8+/
JgULR8VlaNDPYA0u/XKiYTwAF1Jev/WXySrvlNtLysa8lPb0CWG3XA5CqATViUYT
V9MCJ5CvMfiQ1k1Q4T4Om1sRWE6GgN6/IxcUtkJ2ZN2d/tt1tUV0CIusdXwEVjXR
7kX0pha/876/C/K65odT0El3V6LClod46sfBbOX1wjr0sBClJ8PnfPlQtBlIhn8n
3ex6IhsNGo7Ip3Mrx4zN5q9wETiLVc1Drag8DkTnkCrLPdXJWVatts5X54BV8aPE
63ZuEKzFb3prNziOgN0F
=JzKd
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

James,

On 11/18/16 2:39 PM, Christopher Schultz wrote:
> James,
> 
> On 11/18/16 2:06 PM, James H. H. Lampert wrote:
>> On 11/18/16, 9:08 AM, Caldarale, Charles R wrote:
>>> No, 8009 is the default port for communication between httpd
>>> and Tomcat.  8005 is the default shutdown port.
> 
>>> Correct.  If you're not using it, remove (or comment out) the 
>>> declaration.
> 
>> Fascinating.
> 
>> Can somebody point me to the right docs, so I can learn more
>> about this?
> 
> There are a couple of resources.
> 
> The first would be to have a look at the stock server.xml that
> comes with Tomcat, since you can find the AJP Connector on port
> 8009 defined in there. I'm not sure there is any documentation on
> the web that says what the default configuration is... because it's
> just so easy to look.
> 
> The second would be the configuration guide's coverage of the AJP 
> connector[1], which will largely be irrelevant to you.
> 
> If you aren't using AJP for anything, you should probably just
> disable it by commenting-out the  element, or deleting
> it entirely.
> 
> AJP is only useful if you have a reverse-proxy out in front of
> Tomcat and you'd like to use AJP as the communication protocol
> between that proxy and Tomcat. No proxy = no AJP, unless you are
> running some weird kind web browser that for some reason
> understands AJP.

Oh, one more thing: if someone is probing your network, you might want
to find out who it is. It might be a basic security scan by a blue
team, or it could be malware scanning the network to see if anything
interesting is available.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYL1wyAAoJEBzwKT+lPKRYr+0P/R3TEm6PYbm7CfN78lZjoN7C
Fq/QqulXpRcMO3HaUlo+Xg4coapDC2BX6u/ck/znNIFjXa7CHlEwglmtZ3UF8qbU
4URHPKvV040gcAYsEqOJmd3P6jCtjo4UN4ZnS58V9K99KnZibt8y3NFnofR9KVaN
axB6OifluDYrlS0q3p+CmRhBPDBsVjajadzR/hoNIm2bjgXkjNAAcNaCHokj1Dbj
xMBwSx6c2jbAWD+GRXGMUDxI3UMa3eRK6rL18JQ2GQ96L13Kk1EjozoXwxctf3+D
5sTRpafMWtMCE4dIBYZ4LIqvoQgfa/sz2ZfDNiSbU1I0Hw6Y+RZ+6VvjuSWcTEEM
dA44Ee3tZaqwuLmhZb1UUDj0o35kRz5z661IUSONmOh9mA4cIbfE7RFqJlJhZiPj
47B0+LO4xbpNsNpR7Rdsz/8HFVFur9LuvcJ2Ta6UFv9Pzj1Nc8gqibRwfGA0dV5t
gEgdZ3uWrIba4c8HkB/NJDvrtG6rkyoRCUesrgjskuoUuC1qbMA3SV5et9vgqMvS
BvQNgsnzXqWEXKOCnRQwI3bDcbtEUfV8EtDDZrhmgBCXGmexf0Fs3KKm0AOS9qOj
zhmbN3E523PJqEj+op4PQLzUoimPWqtXboKX0qkiqZek92YptHq7ydOoJ87PmJRw
OqGjeldxAdDc8fnzXvyc
=ey8d
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat6 enabling SSL for JDBC during startup

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Kumar,

On 11/16/16 5:01 PM, Mohan Kumar wrote:
> I'm using Tomcat 6.0.29 version with Java 1.6.0_45 on Windows
> server 2008 and started noticing SSL related information in
> Catalina logs recently.

Upgrade upgrade upgrade! All 3 of those components are hideously old.

>> From the logs it looks like SSL is getting enabled for JDBC. Any
>> help to
> understand why the below information shows up in catalina logs
> would be much appreciated? There is nothing that got changed in the
> JDBC datasource configuration recently and the application uses SQL
> server 2008.
> 
>> From catalina.log
> 
> Oct 31, 2016 10:18:26 AM com.microsoft.sqlserver.jdbc.TDSChannel
> enableSSL INFO: java.security path: C:\Program
> Files\Java\jre6\lib\security Security providers: [SUN version 1.6,
> SunRsaSign version 1.5, SunJSSE version 1.6, SunJCE version 1.6,
> SunJGSS version 1.0, SunSASL version 1.5, XMLDSig version 1.0,
> SunPCSC version 1.6, SunMSCAPI version 1.6] SSLContext provider
> info: Sun JSSE provider(PKCS12, SunX509 key/trust factories, SSLv3,
> TLSv1)

It looks like the MS JDBC driver is logging some stuff at the INFO log
level. I wouldn't have put that at the INFO level, but ... whatever.

> Oct 31, 2016 10:18:26 AM org.apache.naming.NamingContext lookup 
> WARNING: Unexpected exception resolving reference 
> org.apache.tomcat.dbcp.dbcp.SQLNestedException: Error preloading
> the connection pool at
> org.apache.tomcat.dbcp.dbcp.BasicDataSource.createDataSource(BasicData
Source.java:1398)
>
> 
at
org.apache.tomcat.dbcp.dbcp.BasicDataSource.getLogWriter(BasicDataSource
.java:1098)
> at
> org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory.createDataSource(Ba
sicDataSourceFactory.java:350)
>
> 
at
org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory.getObjectInstance(Bas
icDataSourceFactory.java:156)
> at
> org.moss.jdj.dbcp.EncryptedDataSourceFactory.getObjectInstance(Encrypt
edDataSourceFactory.java:25)
>
> 

> at java.lang.reflect.Method.invoke(Unknown Source) at
> org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at
> org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) 
> Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The
> driver could not establish a secure connection to SQL Server by
> using Secure Sockets Layer (SSL) encryption. Error: "SQL Server
> returned an incomplete response. The connection has been closed.".

Looks like your JDBC driver can't make a secure connection to the SQL
Server instance.

Are you expecting to use TLS to connect to your db?

Have you checked to see if your client's or server's certificates have
expired or have some other problem? Perhaps you are being MITMd.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYL1vuAAoJEBzwKT+lPKRYnVUP/iZQzcSMvP8U0VuEg2EIocd2
YWi1AkfefbQo05QEdJLbP5Rn3Al5LWo2UUzR/HfsHMrxSNdScoqQjxJqSOxdiyy7
0ykHPjWX9ub0awkev0A/Fx3Cz/t21rzpopD55lAorEryju6Vc/S2Ce8aul9+iZgN
0Ti2tnrcfdpsZRrRTfPpx0uUuD2Fb/ZcclnzqAMF0YaDZsy32uOUVRSy9OF6MSYs
dZW9eJYeC7Fj43YjUsbCzaITEuCE0hqTuE610F+bQ2vUETeuwkedVkw5AFcamwvk
QN39/arSV94Cd+r8CQ+Z1tE8fac1pW3Jo3P95sHau04YEXyG9nb1WW+bWSksWL8S
6493phRwrd8+nb1uDo0s4my0+yjJdstrsRI2QjwDHgHsMm85NJ88AKdIcX4mEkS8
HvAKYK8FI0JekuArhIXmFybJh6bbhJqQUCl4FsC5Y/ki32j+EI3Fha+xXOR8OmA+
9WL6TP+IZcFpZJ2B8rm+47nn2VjjAa1vEXl6qwOY3UqsIduSWHNf/8VE25K+QEm2
7cyELCQlmDoZSsvXC36JrPeoWlD2dItZQmmvSYKX9aqooOGXK9ZnZ0ZbTWfXGdub
p4bqye+jX8A6RSNu5LsEimewkOJljd3Pj2T8tZKUZFV6mR6/Vz4mvH+fK90rcuwB
0VyFEZQkMbD6Onnhozlo
=nC98
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

James,

On 11/18/16 2:06 PM, James H. H. Lampert wrote:
> On 11/18/16, 9:08 AM, Caldarale, Charles R wrote:
>> No, 8009 is the default port for communication between httpd and 
>> Tomcat.  8005 is the default shutdown port.
> 
>> Correct.  If you're not using it, remove (or comment out) the 
>> declaration.
> 
> Fascinating.
> 
> Can somebody point me to the right docs, so I can learn more about
> this?

There are a couple of resources.

The first would be to have a look at the stock server.xml that comes
with Tomcat, since you can find the AJP Connector on port 8009 defined
in there. I'm not sure there is any documentation on the web that says
what the default configuration is... because it's just so easy to look.

The second would be the configuration guide's coverage of the AJP
connector[1], which will largely be irrelevant to you.

If you aren't using AJP for anything, you should probably just disable
it by commenting-out the  element, or deleting it entirely.

AJP is only useful if you have a reverse-proxy out in front of Tomcat
and you'd like to use AJP as the communication protocol between that
proxy and Tomcat. No proxy = no AJP, unless you are running some weird
kind web browser that for some reason understands AJP.

- -chris

[1] http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYL1kHAAoJEBzwKT+lPKRYIOcP/35mDYqAGqLbpCprbz++u1z8
qI0o9jskt1ksaicyD08bI+zpFOvU1dd4wRtipMuExcFyrkNKnuIsIgPGw4PO5zBN
D1v75QsSmsivC3Z4/xaN3rbAL+La8WcwfOP17uww9uEVk7kEFS4iBu8U0fdV2DPw
VglX9bch8S0WR2PlcrpaEobDpH9R8jQQuxig9GhXEN6j9SX+q/qgPRZweXCcFERJ
p+fbyhDQTFHJzqjrJ+aYly2/4HfvHJtgsXQc006Vy6eq/QMXjQLykA+B9RpXAW+N
74Ayk26O9l52vvupvYqaJwLB1ExwHz89WQRyPWvQL2OpwI+n82uGtnOkiSREq7t4
Nrlfy/PAIJnpB2tVkKTlvNm7wmF0rt4+3JUlM7UA7cT5SKBhQkYRGGlFLpnaTsdW
wPM45gWOf/ANSFt00Mv0uLIIw1JUUHOOtBs1fSiwiRr7EMsoXHxyiRJefZ1fU+gP
yyCtcY6oiEmw2JpbrbSrG7tkDUR/GpVDoBRNPMIH4uR6K53xA6zxBUbWP7vBIQPr
2z+R3hVURy9I78VNr2Ox3GBTpTEg6gaCiLgxIACzWvyE47QRI/jJ1naZzsEo4sav
l3m4iCLKu6OA8vWQk26SsIu/Ugs1lJM4kZQ5LJx9Rt43apQ6HLrSraPEOW0SKr8O
kgIr7aodnkRVnYURregX
=czD/
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[Caldarale, Charles R]

> From: James H. H. Lampert [mailto:jam...@touchtonecorp.com] 
> Subject: Re: Getting "Invalid message received with signature x" messages 
> in catalina.out

> > No, 8009 is the default port for communication between httpd and Tomcat.  
> > 8005 is 
> > the default shutdown port.

> > If you're not using it, remove (or comment out) the declaration.

> Can somebody point me to the right docs, so I can learn more about this?

Start with the FAQ:
https://wiki.apache.org/tomcat/FAQ/Connectors

Then the official places:
http://tomcat.apache.org/tomcat-8.5-doc/config/http.html
http://tomcat.apache.org/tomcat-8.5-doc/config/ajp.html

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Sanity Check

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Konstantin,

On 11/18/16 2:10 PM, Konstantin Kolinko wrote:
> One more authority, that I forgot to mention in my mail: IANA
> registry of mime types
> 
> Registry: 
> https://www.iana.org/assignments/media-types/media-types.xhtml
> 
> Registration entry for "application/x-www-form-urlencoded" 
> https://www.iana.org/assignments/media-types/application/x-www-form-ur
lencoded
>
>  -> Encoding considerations : 7bit
> 
> According to RFC defining this registry, it means that the data is 
> 7-bit ASCII only. https://tools.ietf.org/html/rfc6838#section-4.8

Oh, that's the nail in the coffin.

application/x-www-form-urlencoded from W3C says "if the character
doesn't fit into the encoding of the message, it must be %-encoded"
but it never says what "the encoding of the message" actually is. My
worry was that it was mutable, and that UTF-8 was a valid encoding,
meaning that 0xc2 0xae on the wire would have been acceptable (rather
than %C2%AE).

If application/a-www-form-urlencoded is *absolutely* supposed to be
7-bit ASCII, then nothing above 0x7f can ever be legally transferred
across the wire when using that content-type.

This solves André's problem with this content-type where he wanted to
specify the charset to be used. It seems the standard defines the
character set: US-ASCII.

The only problem now is that it's not clear how to turn %C2%AE into a
character because you have to know that UTF-8 and not Shift-JS or
whatever is being used.

> -> Required parameters : No parameters -> Optional parameters :  No
> parameters
> 
> OK. So no charset= parameter is allowed. My advise to specify the
> charset parameter was wrong.

Agreed: it is always against the spec(s) to specify a charset for any
MIME type that is not text/*.

> Though historically ~10 years ago I saw 
> "application/x-www-form-urlencoded;charset=UTF-8" Content-Type in
> the wild.

Oh, I'm sure you saw it. I even tossed that into my client to see if
it would make a difference. Not surprisingly, it did not.

> It was a web site authored in WML (Wireless Markup Language) and 
> accessed via WAP protocol by mobile phones.
> 
> (Specification reference for this WML/WAP usage: 
> http://technical.openmobilealliance.org/Technical/release_program/docs
/Browsing/V2_3-20070227-C/WAP-191-WML-2219-a.pdf
>
>  Document title: WAP WML WAP-191-WML 19 February 2000
> 
> Wireless Application Protocol Wireless Markup Language
> Specification Version 1.3
> 
> -> Page 30 of 110 (in Section "9.5.1 The Go Element"): There is a
> table, where the following line is relevant:
> 
> Method: post Enctype: application/x-www-form-urlencoded Process:
> [...] The Content-Type header must include the charset parameter to
> indicate the character encoding.
> 
> I suspect that the above URL is not the official location of the 
> document. I found it through Googling. Official location should be
> http://www.wapforum.org/what/technical.htm )
> 
> 
> Apache Tomcat supports the use of charset parameter with
> Content-Type application/x-www-form-urlencoded in POST requests.

Interesting. I suspect that's because there are practical situations
where "being liberal with what you accept" is more appropriate than
angrily demanding that all clients be 100% spec-compliant :)

The (illegal) charset parameter can only mean one thing: the character
encoding to use to assemble url-decoded bytes into an actual string
value (e.g. %C2%AE -> 0xc2 0xae -> "®" when using UTF-8).

Thanks for that final reference; it really does close the case on this
whole thing.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYL1YQAAoJEBzwKT+lPKRYyAkP/3Udkqjiqa7BhRH2Gxo8WhNf
Wm7BbWGS8vlgbHH/0mNzFPSxGi7mWxlimaGnc+H8fqk54RZCeNaqQPqPXhG7ldA1
QtR/1H1kXoqUNFmqnj3FBgA6UBZhql9RyLZLbeHdZMK9i1sN4bI/CEa2EP5rZ+0d
0sXXj8wRz+yk2bXtdyuW8yHzQRNB/+XJbOrQBVqc+u//K/+q9I8eEN0SlZo8+9t2
9hqqcufhd9YtuH1Ypn1M73l72WFWad7BEgPPG+noLcB8/OrSXfeF2ELEe9dzv6r6
Jyxas6uUiplE8+/1QTu8MYSGqeo3l/xgixCD9gEMLNFBlcLPlQcRhaoQ08bgZOcT
SyzVIYYCL7R7MsB1f3QFDEax0vwIi0a6Zrfaa3oqklXEhNuVk/Ani8+sbFw01iHW
ZxV6vc0v9APMOg3jVQug3UC1kAGcZi8toISKyrFt9lwK0AbDrSVKfe4sKql91yQm
wQCG3e/RjoSo1LEmh9yszurNtOy2ecqTBkIS2cksf4crYSqpefCyB/GpnrJaHMvx
P/PQ0hVZUg05Z/tj7Dxma5mWrlm9IQBC+inDiwIEnl9hGp67KfxZAEk8hUstDBWw
AK78+DsseGpyx40o6scDz8dR9ThnTHm3k0zhdUZoORwfft78Ar0HYjZCDQArhuMK
BDGqIegIrNeJtCDnYOdq
=nJCy
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Chuck,

On 11/18/16 12:08 PM, Caldarale, Charles R wrote:
>> From: James H. H. Lampert [mailto:jam...@touchtonecorp.com] 
>> Subject: Re: Getting "Invalid message received with signature
>> x" messages in catalina.out
> 
>> This is interesting: > redirectPort="8443" />
> 
>> Isn't 8009 some sort of backchannel control port, perhaps the one
>> used for controlled shutdown of Tomcat?
> 
> No, 8009 is the default port for communication between httpd and
> Tomcat.  8005 is the default shutdown port.
> 
>> It seems to be defined as an AJP port "straight out of the box,"
>> is also so-defined on our own Tomcat server, and is presumably
>> so-defined at all our other customer installations.
> 
> Correct.  If you're not using it, remove (or comment out) the
> declaration.
> 
> 
>> And yet this is the first time I've ever seen these error
>> messages.
> 
> Something with access to your network is probing that port.

+1

And they aren't speaking AJP, which is why you are getting that error.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYL1PWAAoJEBzwKT+lPKRYqLoP/iqvju0oCJdz6KNLeLpmQrFP
MmGFfDmGxjox1HHx7ApTC2jq1/q0yYlsqafCuQ1kDrkBbqfr10LI7IwWkfakKEcB
pEZakhyt5Pi6uGJjjM2LuoDBSJ/KBYC391MqENcsrm7fFJzfjVIxjsl15YGYiko/
BR2ghLUiH0Qy2H/KzzmWFdEtUf5eeUQMMcdCpctcm9+RPjuiJ/cJXwQIt+M8wLGV
KVt7OtgX0ly+4n7Msy+yDFDn/T/cdKq6BigAXhqXl0Ho0bKztolryb9FbQJ7unB+
Lkug3/z/EekoftgyxZBFBe9+6XTJKFMuesCzPWij9K9DgKFozyUy9rqfZMUCpyzg
v8u6khJHNIGSONmHNISlcj1yeeqNxW800ZYzsHyxfU6JZTVAvC2JxdPKaZG5amXi
/OjiN8q6tYeHD3xF0YkQixSTjzjka+2AyawKECUKu5GLl1Sxpt4MDCtJu1PX5X8C
GiNxSPY8fMGCNnV4AetSIK4YOOWjsxvkomAeKFgEvpb7NdokFRG6If0GF4Z0O1DG
ZP5QPRz6g9vmk26+MPHsSh4KBT3yj8iJag5f5le4NqYJ8AeNL4BaKWoqRdHpACq1
u9SoowjOgJfQNjI/QAs0V14bQYIlLgUN2e2QMtJ0hfD4+1f9OgmYW6kd1zNWF+W8
MHIy7SogtXLH5IXU4biO
=W6NW
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Sanity Check

[Konstantin Kolinko]

2016-11-18 19:02 GMT+03:00 Christopher Schultz :
> André,
>
> On 11/18/16 3:50 AM, André Warnier (tomcat) wrote:
>> On 18.11.2016 05:56, Christopher Schultz wrote:
>>> Since UTF-8 is supposed to be the "official" character encoding,
>>
>> Now where is that specified ?  As far as I know, the default
>> charset for everything HTTP and HTML-wise is still iso-8859-1, no ?
>> (and unfortunately so).
>
> I apologize for the sloppy language: this particular vendor's service
> claims that UTF-8 if the standard *for their service*. Not for HTTP in
> general.
>
>>> The vendor has responded with (paraphrasing) "it seems we don't
>>> completely follow this standard; we're considering what to do
>>> next, which may include no change". This is a big vendor with
>>> *lots* of software clients, so maintaining backward compatibility
>>> is going to be a big deal for them. I've got some tricks up my
>>> sleeve if they decide not to change anything. Hooray for specs.
>>> :(
>>
>> What I never understood in all that, is why browsers and other
>> clients never seem to respect (and servers do not seem to enforce)
>> what is indicated here :
>>
>> https://www.ietf.org/rfc/rfc2388.txt 4.5 Charset of text in form
>> data
>>
>> This would be a simple way to get rid of umpteen character
>> set/encoding issues encountered when trying to interpret 
>> data POSTed to web applications.
>
> The problem is that application/x-www-form-urlencoded doesn't give a
> client a natural way to specify the character encoding, and a/xwfu can
> be used inside of a multipart/form-data package as well. You've just
> moved the problem from the Content-Type of the request to the
> Content-Type of the *part* of the multi-part request. Nothing has been
> solved by using multipart/form-data.
>
> And browsers certainly DO use that, but almost exclusively for things
> like file-upload, since files tend to be very big already, and
> urlencoding a bunch of binary bytes makes the file size increase quite
> a bit.
>
>> It seems to me contrary to common sense that in our day and age,
>> the rules for this could not be set once and for all to something
>> like :
>>
>> 1) the default character set/encoding of HTTP and HTML is
>> Unicode/UTF-8 (instead of the current really archaic iso-8859-1) 2)
>> URLs (including query-strings) should be by default interpreted as
>> Unicode/UTF-8, encoded as per
>> https://tools.ietf.org/html/rfc3986#section-2 3) for POST requests
>> : - for the Content-type "application/x-www-form-urlencoded",
>> there SHOULD be a charset attribute indicating the charset and
>> encoding. By default, this is "text/plain; charset=UTF-8"
>
> Don't forget, charset == encoding. The text/plain is the MIME type,
> and that's already been defined as application/x-www-form-urlencoded.
> Somewhere it should just explicitly say "a/xwfu" must contain only
> ASCII bytes, and always encodes a text blob in UTF-8 encoding.
>
> But it will never happen (see below).

One more authority, that I forgot to mention in my mail:
IANA registry of mime types

Registry:
https://www.iana.org/assignments/media-types/media-types.xhtml

Registration entry for "application/x-www-form-urlencoded"
https://www.iana.org/assignments/media-types/application/x-www-form-urlencoded

-> Encoding considerations : 7bit

According to RFC defining this registry, it means that the data is
7-bit ASCII only.
https://tools.ietf.org/html/rfc6838#section-4.8

-> Required parameters : No parameters
-> Optional parameters :  No parameters

OK. So no charset= parameter is allowed.
My advise to specify the charset parameter was wrong.

Though historically ~10 years ago I saw
"application/x-www-form-urlencoded;charset=UTF-8" Content-Type in the
wild.

It was a web site authored in WML (Wireless Markup Language) and
accessed via WAP protocol by mobile phones.

(Specification reference for this WML/WAP usage:
http://technical.openmobilealliance.org/Technical/release_program/docs/Browsing/V2_3-20070227-C/WAP-191-WML-2219-a.pdf

Document title:
WAP WML
WAP-191-WML
19 February 2000

Wireless Application Protocol
Wireless Markup Language Specification
Version 1.3

-> Page 30 of 110 (in Section "9.5.1 The Go Element"):
There is a table, where the following line is relevant:

Method: post
Enctype: application/x-www-form-urlencoded
Process: [...] The Content-Type header must include the charset
parameter to indicate the character encoding.

I suspect that the above URL is not the official location of the
document. I found it through Googling.
Official location should be http://www.wapforum.org/what/technical.htm
)


Apache Tomcat supports the use of charset parameter with Content-Type
application/x-www-form-urlencoded in POST requests.

>> - for the Content-type "multipart/form-data", each "part" MUST have
>> a Content-type header.  If this Content-type is a "text" type, then
>> the Content-type header SHOULD contain a charset attribute. If
>> omitted, by default this is 

Re: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[James H. H. Lampert]

On 11/18/16, 9:08 AM, Caldarale, Charles R wrote:

No, 8009 is the default port for communication between httpd and Tomcat.  8005 
is the default shutdown port.



Correct.  If you're not using it, remove (or comment out) the declaration.


Fascinating.

Can somebody point me to the right docs, so I can learn more about this?

--
JHHL

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[Caldarale, Charles R]

> From: James H. H. Lampert [mailto:jam...@touchtonecorp.com] 
> Subject: Re: Getting "Invalid message received with signature x" messages 
> in catalina.out

> This is interesting:
>  

> Isn't 8009 some sort of backchannel control port, perhaps the one used 
> for controlled shutdown of Tomcat?

No, 8009 is the default port for communication between httpd and Tomcat.  8005 
is the default shutdown port.

> It seems to be defined as an AJP port "straight out of the box," is also 
> so-defined on our own Tomcat server, and is presumably so-defined at all 
> our other customer installations.

Correct.  If you're not using it, remove (or comment out) the declaration.


> And yet this is the first time I've ever seen these error messages.

Something with access to your network is probing that port.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[tomcat]

On 18.11.2016 17:56, James H. H. Lampert wrote:

On 11/17/16, 9:00 PM, Christopher Schultz wrote:


There must be another connector. This one uses HTTP, and the error
messages you posted are using AJP. Do you have a connector in
conf/server.xml that you thought was disabled, but isn't?


Dear Mr. Schultz:

This is interesting:

 


Isn't 8009 some sort of backchannel control port, perhaps the one used for 
controlled
shutdown of Tomcat?

It seems to be defined as an AJP port "straight out of the box," is also 
so-defined on our
own Tomcat server, and is presumably so-defined at all our other customer 
installations.

And yet this is the first time I've ever seen these error messages.

Now, I'm more puzzled than ever about this.



My own guess : at the date at which these messages started to happen, some additional 
(monitoring ?) program was started in your network, which sends HTTP messages to any 
server port it finds listening.  Only in this case, it sends HTTP messages to a port which 
does not understand HTTP, so it complains.




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Sanity Check

[tomcat]

On 18.11.2016 17:02, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

André,

On 11/18/16 3:50 AM, André Warnier (tomcat) wrote:

On 18.11.2016 05:56, Christopher Schultz wrote:

Since UTF-8 is supposed to be the "official" character encoding,


Now where is that specified ?  As far as I know, the default
charset for everything HTTP and HTML-wise is still iso-8859-1, no ?
(and unfortunately so).


I apologize for the sloppy language: this particular vendor's service
claims that UTF-8 if the standard *for their service*. Not for HTTP in
general.


The vendor has responded with (paraphrasing) "it seems we don't
completely follow this standard; we're considering what to do
next, which may include no change". This is a big vendor with
*lots* of software clients, so maintaining backward compatibility
is going to be a big deal for them. I've got some tricks up my
sleeve if they decide not to change anything. Hooray for specs.
:(


What I never understood in all that, is why browsers and other
clients never seem to respect (and servers do not seem to enforce)
what is indicated here :

https://www.ietf.org/rfc/rfc2388.txt 4.5 Charset of text in form
data

This would be a simple way to get rid of umpteen character
set/encoding issues encountered when trying to interpret 
data POSTed to web applications.


The problem is that application/x-www-form-urlencoded doesn't give a
client a natural way to specify the character encoding,


Yes, it does.  In the case of this content-type, the whole list of posted parameters is 
provided as one big chunk of text, in the body of the request.
The content-type "application/x-www-form-urlencoded" implies text, because there is no 
good way in that format to include any post parameter which is not text.
Since it is text, there is no good reason why the (single) Content-type header of the POST 
could not provide a charset attribute.


 and a/xwfu can

be used inside of a multipart/form-data package as well. You've just
moved the problem from the Content-Type of the request to the
Content-Type of the *part* of the multi-part request. Nothing has been
solved by using multipart/form-data.


I have no changed or moved anything. I have just been adding the requirement that if any 
of these parts concerns a text-type part, it SHOULD also contain a charset attribute.


This is precisely what browsers do not do, for whatever reason which is beyond my 
comprehension.  The parts already have a Content-type. It is just the charset attribute 
*for the parts which are text* that is missing, despite what the rfc2388 recommendation says.




And browsers certainly DO use that, but almost exclusively for things
like file-upload, since files tend to be very big already, and
urlencoding a bunch of binary bytes makes the file size increase quite
a bit.


It seems to me contrary to common sense that in our day and age,
the rules for this could not be set once and for all to something
like :

1) the default character set/encoding of HTTP and HTML is
Unicode/UTF-8 (instead of the current really archaic iso-8859-1)


2)

URLs (including query-strings) should be by default interpreted as
Unicode/UTF-8, encoded as per
https://tools.ietf.org/html/rfc3986#section-2


3) for POST requests

: - for the Content-type "application/x-www-form-urlencoded",
there SHOULD be a charset attribute indicating the charset and
encoding. By default, this is "text/plain; charset=UTF-8"


Don't forget, charset == encoding. The text/plain is the MIME type,
and that's already been defined as application/x-www-form-urlencoded.


I made a mistake here. Scratch the "text/plain;" part above. The charset attribute should 
be added to the existing Content-type header.

In other words, the header should be :
Content-type: application/x-www-form-urlencoded; charset=
The MIME type "x-www-form-urlencoded" already *implies* that this is text, 
URL-encoded.
It just fails to specify what charset/encoding the query string was encoded with, *before* 
it was URL-encoded.



Somewhere it should just explicitly say "a/xwfu" must contain only
ASCII bytes, and always encodes a text blob in UTF-8 encoding.

But it will never happen (see below).


- for the Content-type "multipart/form-data", each "part" MUST have
a Content-type header.  If this Content-type is a "text" type, then
the Content-type header SHOULD contain a charset attribute. If
omitted, by default this is "charset=UTF-8".

and be done with it once and for all.


Right: once and for all, for new clients who implement the spec. All
old clients, servers, proxies, , etc. be damned. It's just not
possible due to the need to be backward-compatible with really weird
stuff like "smart" toasters and refrigerators, WebTV (remember that?)
and all manner of embedded devices that will never be updated.

What we really need is a new header that says "here's everything you
need to know about encoding for this request"


There is no need for a new header. The existing 

Re: Getting "Invalid message received with signature xxxxx" messages in catalina.out

[James H. H. Lampert]

On 11/17/16, 9:00 PM, Christopher Schultz wrote:


There must be another connector. This one uses HTTP, and the error
messages you posted are using AJP. Do you have a connector in
conf/server.xml that you thought was disabled, but isn't?


Dear Mr. Schultz:

This is interesting:

 


Isn't 8009 some sort of backchannel control port, perhaps the one used 
for controlled shutdown of Tomcat?


It seems to be defined as an AJP port "straight out of the box," is also 
so-defined on our own Tomcat server, and is presumably so-defined at all 
our other customer installations.


And yet this is the first time I've ever seen these error messages.

Now, I'm more puzzled than ever about this.

--
JHHL

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Sanity Check

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

André,

On 11/18/16 3:50 AM, André Warnier (tomcat) wrote:
> On 18.11.2016 05:56, Christopher Schultz wrote:
>> Since UTF-8 is supposed to be the "official" character encoding,
> 
> Now where is that specified ?  As far as I know, the default
> charset for everything HTTP and HTML-wise is still iso-8859-1, no ?
> (and unfortunately so).

I apologize for the sloppy language: this particular vendor's service
claims that UTF-8 if the standard *for their service*. Not for HTTP in
general.

>> The vendor has responded with (paraphrasing) "it seems we don't 
>> completely follow this standard; we're considering what to do
>> next, which may include no change". This is a big vendor with
>> *lots* of software clients, so maintaining backward compatibility
>> is going to be a big deal for them. I've got some tricks up my
>> sleeve if they decide not to change anything. Hooray for specs.
>> :(
> 
> What I never understood in all that, is why browsers and other
> clients never seem to respect (and servers do not seem to enforce)
> what is indicated here :
> 
> https://www.ietf.org/rfc/rfc2388.txt 4.5 Charset of text in form
> data
> 
> This would be a simple way to get rid of umpteen character
> set/encoding issues encountered when trying to interpret 
> data POSTed to web applications.

The problem is that application/x-www-form-urlencoded doesn't give a
client a natural way to specify the character encoding, and a/xwfu can
be used inside of a multipart/form-data package as well. You've just
moved the problem from the Content-Type of the request to the
Content-Type of the *part* of the multi-part request. Nothing has been
solved by using multipart/form-data.

And browsers certainly DO use that, but almost exclusively for things
like file-upload, since files tend to be very big already, and
urlencoding a bunch of binary bytes makes the file size increase quite
a bit.

> It seems to me contrary to common sense that in our day and age,
> the rules for this could not be set once and for all to something
> like :
> 
> 1) the default character set/encoding of HTTP and HTML is
> Unicode/UTF-8 (instead of the current really archaic iso-8859-1) 2)
> URLs (including query-strings) should be by default interpreted as 
> Unicode/UTF-8, encoded as per
> https://tools.ietf.org/html/rfc3986#section-2 3) for POST requests
> : - for the Content-type "application/x-www-form-urlencoded",
> there SHOULD be a charset attribute indicating the charset and
> encoding. By default, this is "text/plain; charset=UTF-8"

Don't forget, charset == encoding. The text/plain is the MIME type,
and that's already been defined as application/x-www-form-urlencoded.
Somewhere it should just explicitly say "a/xwfu" must contain only
ASCII bytes, and always encodes a text blob in UTF-8 encoding.

But it will never happen (see below).

> - for the Content-type "multipart/form-data", each "part" MUST have
> a Content-type header.  If this Content-type is a "text" type, then
> the Content-type header SHOULD contain a charset attribute. If
> omitted, by default this is "charset=UTF-8".
> 
> and be done with it once and for all.

Right: once and for all, for new clients who implement the spec. All
old clients, servers, proxies, , etc. be damned. It's just not
possible due to the need to be backward-compatible with really weird
stuff like "smart" toasters and refrigerators, WebTV (remember that?)
and all manner of embedded devices that will never be updated.

What we really need is a new header that says "here's everything you
need to know about encoding for this request" and clients and servers
who both support that header can use it. All other uses need to
fall-back to this old and nasty heuristic.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJYLyYAAAoJEBzwKT+lPKRYF90QAJNOyadgrG7DDyWLSuFfKkep
VAoc5yziddaHoTKpcExGrEB+LV5gJ35XR2Q+CiOCNoTR1O3oOJyflk2s8e+lqeZ9
2rqIlauOwwWC13dfwpcOENkeC3eyHn85d3NkuuFsqvqRl+Wuv4qvqRiv/kos723i
cKmgqbAE9zRjNxuIqym3J8m6BhwzJGN3HqtiUueTYphChW81V10hc8XElJEPDbAH
eGpdunp8eu4pbi36RZV5r2nZU2yHZVDd+HJnTFG4WJ/NvHODuJsR39fB+GANI0QJ
+OHS9b7Wpcl2eCPs8geVTSqe57vDBrhymFjIUorPuQeW0SxrwDJMdTJ4zYtqnY2B
fD7u9Lvo+RT/eskIcdFGVq5xUEBr2OIfx2XO2V7VlA52x+WJ421TLFRUQq67Un40
yDsPXEBHMVar2cyG2wOJsb/t6ndlCY30b1FPOD2zrg1XFxxzjaOCwUtZXqgX7sfu
H1Dalbg4S/8vPS5Yrd7ZHk4RgYr5GGMBcK01KC07Q/TrOFkw9ssqvfQTyl30jxZ/
/x74KMRAbJVsUuhJ0i8QLM0KqPMpJ9wP9jwQF4YFUFwTDp6xBa/FRVAXCmJQxKom
JFCky4YhVvOGVOK2iwDDQRJee1ahz0V+maJii1fSHVYMCrWrzGNZ6LMeuZAsovs0
ZjotO2X+XAPpLwczn6tI
=7oxR
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Sanity Check

[tomcat]

On 18.11.2016 05:56, Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Konstantin,

On 11/17/16 4:58 PM, Konstantin Kolinko wrote:

2016-11-17 17:21 GMT+03:00 Christopher Schultz
:

All,

I've got a problem with a vendor and I'd like another opinion
just to make sure I'm not crazy. The vendor and I have a
difference of opinion about how a character should be encoded in
an HTTP POST request.

The vendor's API officially should accept requests in UTF-8
encoding. We are using application/x-www-form-urlencoded content
type.

I'm trying to send a message with a non-ASCII character -- for
example, a ® (that's (R), the registered trademark symbol).

The Java code being used to package-up this POST looks something
like this:

OutputStream out = httpurlconnection.getOutputStream();
out.print("notetext="); out.print(URLEncoder.encode("test®",
"UTF-8")); out.close();

So the POST payload ends up being notetext=test%C2%AE or, on the
wire, the bytes are 6e 6f 74 65 74 65 78 74 3d 74 65 73 74 25 43
32 25 41 45.

The final bytes 25 43 32 25 41 45 are the characters % C 2 % A
E.

Can someone verify that I'm encoding everything correctly?

The vendor is claiming that ® can be sent "directly" like one
might do using curl:

$ curl -d 'notetext=®' [url]

and the bytes on the wire are 6e 6f 74 65 74 65 78 74 3d c2 ae
(note that c2 and ae are "bare" and not %-encoded).


1. That is a wrong way to use curl.  The manual says that the
argument to -d should be properly urlencoded. The above value is an
incorrect one.

https://curl.haxx.se/docs/manual.html See "POST (HTTP)" and below.


+1

The curl manual says that -d is the same as --data-ascii, which is
totally wrong here if they are accepting UTF-8.


2. If you are submitting data programmatically, I wonder why you
are using simple "application/x-www-form-urlencoded".

I think it would be better to use explicit charset argument in the
Content-Type value, as it is easy to do so with Java clients.


Their API expects application/x-www-form-urlencoded. Everything else
they do is in JSON... I have no idea why they don't accept JSON as
input, but that's the deal.

MIME types that aren't text/* aren't supposed to have Content-Type
parameters.


Maybe more precisely : there SHOULD be a Content-type header; but a "charset" attribute 
only makes sense if the content type is, generally-speaking, "text".
("text/plain" certainly qualifies; but one may argue about "text/html" and variants e.g., 
since these formats may have their own embedded charset indications)





3. The application/x-www-form-urlencoded encoding was originally
specified in HTML specification.

Current specification:
https://www.w3.org/TR/html51/sec-forms.html#urlencoded-form-data

It defers details to
https://url.spec.whatwg.org/#concept-urlencoded-serializer

Historic, HTML 4.01:
https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.1


All true, but the spec argues with itself over the character encoding,
and browsers make this worse with their stupid "I'll use whatever
character encoding was used to load the page containing the form"
behavior. With a software-client API, there basically is no spec.

Their assertion is that their character encoding "is UTF-8". But it
looks like they aren't doing it right.


My opinion is that the correct value on the wire is 25 43 32 25 41
45 = % C 2 % A E.


So, the same bytes as I had, right?


If a vendor accepts non-encoded "c2 ae": it technically may work
(in some versions of some software), but this is not a standard
feature and one would better not rely on it.

Technically, if non-encoded bytes ("c2 ae") are accepted, they
won't be confused with special character ("=", "&", "+", "%",
CRLF), as all multi-byte UTF-8 characters have 0x80 bit set.


Their non-%-encoded bytes could be considered legitimate, because the
application/x-www-form-urlencoded rules say that any character "in the
character set of the request" can be dropped-into the request without
being %-encoded. But they we are back to the problem of not knowing
what the encoding of the request is.

Since UTF-8 is supposed to be the "official" character encoding,


Now where is that specified ?  As far as I know, the default charset for everything HTTP 
and HTML-wise is still iso-8859-1, no ? (and unfortunately so).


 I

would expect that a properly-encoded request would contain nothing but
valid ASCII characters, which means that 0xc2 0xae need to be
%-encoded to become "%c2%ae".


4. You code fragment is broken and won't compile: there are none
"print" methods in java.io.OutputStream.

OutputStream works with byte[] and the method name is "write".


Yes, it was hastily-typed from memory. The true code compiles and runs
as expected.


5. Wikipedia:
https://en.wikipedia.org/wiki/Percent-encoding#The_application.2Fx-www

- -form-urlencoded_type


  Wikipedia mentions XForms spec, ->
https://www.w3.org/TR/2007/REC-xforms-20071029/#serialize-urlencode