Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jason,

On 3/10/16 4:40 PM, Jason Overland wrote:
> Chris,
> 
> On Thu, Mar 10, 2016 at 6:18 AM, Christopher Schultz 
>  wrote:
>> Give this patch a try: ... I have no idea how the options get
>> parsed; we'll see if this simple implementation will get you
>> going again.
>> 
>> -chris
>> 
> 
> The parsing is working correctly.  After applying the patch I
> could login successfully.  Then I added digest=SHA to jaas.config
> and it stopped working ("wrong password").
> 
> On further inspection I found that our CallbackHandler was
> digesting the password before passing it back to the
> JAASMemoryModule, however CredentialHandler expects
> inputCredentials to be plaintext.  So I commented out the part of
> our CallbackHandler that digests the password and now it's working.
> That seems ok to me.

You could also just not specify the "digest=SHA" in your JAAS
configuration. My patch will always configure an otherwise-empty
MessageDigestCredentialHandler which, in the absence of any "digset"
being set, simply use plaintext passwords. (Or, rather, the
CredentialHandler won't actually mutate the credentials on the way
through.)

> So I think this patch is sufficient to get us going again.  Thanks
> for the quick turnaround.  If this patch looks good to everyone, do
> you think it can make it into the next Tomcat patch release?

I see Mark already replied saying he would work on it a bit. My patch
was a quick POC to see if it would work and not, IMO,
production-quality, etc. But we should be able to do something for you
pretty quickly. And you should be able to use it in YOUR production
system right away if you'd like. The final patch will simply be more
robust, support more options, and likely cover cases outside what you
were requesting.

Sorry about the oversight in the MemoryRealm.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlbh80IACgkQ9CaO5/Lv0PCudACguQJO6uAmfKG7pG23McHvJVm/
mA8AnAubCHy73F81KAdLfDdEihffkCQe
=xfMg
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Mark Thomas]

On 10/03/2016 21:40, Jason Overland wrote:
> Chris,
> 
> On Thu, Mar 10, 2016 at 6:18 AM, Christopher Schultz
>  wrote:
>> Give this patch a try:
>> ...
>> I have no idea how the options get parsed; we'll see if this simple
>> implementation will get you going again.
>>
>> -chris
>>
> 
> The parsing is working correctly.  After applying the patch I could
> login successfully.  Then I added digest=SHA to jaas.config and it
> stopped working ("wrong password").
> 
> On further inspection I found that our CallbackHandler was digesting
> the password before passing it back to the JAASMemoryModule, however
> CredentialHandler expects inputCredentials to be plaintext.  So I
> commented out the part of our CallbackHandler that digests the
> password and now it's working.  That seems ok to me.
> 
> So I think this patch is sufficient to get us going again.  Thanks for
> the quick turnaround.  If this patch looks good to everyone, do you
> think it can make it into the next Tomcat patch release?

Great. I've been testing a patch along the same lines locally and see
the same results. I'll get it committed before the next 8.0.x release.

Mark


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Jason Overland]

Chris,

On Thu, Mar 10, 2016 at 6:18 AM, Christopher Schultz
 wrote:
> Give this patch a try:
> ...
> I have no idea how the options get parsed; we'll see if this simple
> implementation will get you going again.
>
> -chris
>

The parsing is working correctly.  After applying the patch I could
login successfully.  Then I added digest=SHA to jaas.config and it
stopped working ("wrong password").

On further inspection I found that our CallbackHandler was digesting
the password before passing it back to the JAASMemoryModule, however
CredentialHandler expects inputCredentials to be plaintext.  So I
commented out the part of our CallbackHandler that digests the
password and now it's working.  That seems ok to me.

So I think this patch is sufficient to get us going again.  Thanks for
the quick turnaround.  If this patch looks good to everyone, do you
think it can make it into the next Tomcat patch release?

Also, yesterday I made a bug on bugzilla.

https://bz.apache.org/bugzilla/show_bug.cgi?id=59154

Thanks again,
Jason

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Christopher Schultz]

Jason,

On 3/9/16 1:19 PM, Jason Overland wrote:
> For what it's worth, that analysis & approach to fixing seem
> reasonable to me.  Yes I'll be glad to file a bug report and test a
> patch.

Give this patch a try:

 CUT =

Index: java/org/apache/catalina/realm/JAASMemoryLoginModule.java
===
--- java/org/apache/catalina/realm/JAASMemoryLoginModule.java   (revision
1725851)
+++ java/org/apache/catalina/realm/JAASMemoryLoginModule.java   (working copy)
@@ -18,6 +18,7 @@

 import java.io.File;
 import java.io.IOException;
+import java.security.NoSuchAlgorithmException;
 import java.security.Principal;
 import java.util.Map;

@@ -221,6 +222,18 @@
 if (options.get("pathname") != null)
 this.pathname = (String) options.get("pathname");

+// TODO: This should probably have many more options available
+MessageDigestCredentialHandler ch = new
MessageDigestCredentialHandler();
+Object digestAlgorithm = options.get("digest");
+if(digestAlgorithm instanceof String) {
+try {
+ch.setAlgorithm((String)digestAlgorithm);
+} catch (NoSuchAlgorithmException nsae) {
+log.error("Cannot initialize credential handler", nsae);
+}
+}
+setCredentialHandler(ch);
+
 // Load our defined Principals
 load();



Once you're re-built, change your JAAS configuration to:

jaas.config:
/** JAAS Login Configuration for the Application **/

JAASTomcat {
   org.apache.catalina.realm.JAASMemoryLoginModule required debug=true
digest=SHA;
};

(note the digest=SHA is on the same line with everything else)

I have no idea how the options get parsed; we'll see if this simple
implementation will get you going again.

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Jason Overland]

Chris,
Thanks for the quick and informative response.

On Wed, Mar 9, 2016 at 7:26 AM, Christopher Schultz
 wrote:

>
>> For authentication our configuration is using a MemoryRealm with
>> digest="SHA".  We are storing usernames and passwords in a
>> tomcat-users.xml file.  We are using a jaas.config which specifies to
>> use a org.apache.catalina.realm.JAASMemoryLoginModule.  We have our
>> own implementation of a CallbackModule.
>
> I must admit I'm not sure how the JAAS configuration fits into all this
> (I simply have no JAAS experience). But I suspect that since
> JAASMemoryLoginRealm extends RealmBase, it needs to be configured similarly.
>
> The  you have in server.xml looks to be configured correctly, but
> I'm not sure it's being used if JAAS is in play. I wouldn't be surprised
> if JAAS is creating a separate instance of the JAASMemoryLoginRealm
> (which is a MemoryRealm) and never setting any of those properties.
>

I was wondering myself how those configurations interacted, if at all.
It does appear as if something like what you're describing is
happening.  That makes me wonder if configuring Realm in server.xml is
necessary at all when JAAS is taking the reigns.

> It looks like the best way to set those properties is via the "options"
> for the realm:
>
>>> jaas.config:
>>> /** JAAS Login Configuration for the Application **/
>>>
>>> JAASTomcat {
>>>org.apache.catalina.realm.JAASMemoryLoginModule required debug=true;
>>> };
>
> Instead of simply "debug=true" for the options, we might want to add
> "digest=SHA" and then handle it in the initializer for JAASMemoryLoginRealm.
>
> I think the reason that this works in the older code is that the default
> algorithm of SHA is built-into the MemoryRealm and doesn't require the
> new CredentialHandler stuff. The new method for creating a
> CredentialHandler without actually specifying one (i.e. digest="SHA")
> requires that the "digest" actually be set to something. Since that's
> not happening, you get an NPE.
>
> Since you are willing to build your Tomcat from source, can I give you a
> patch to test?
>
> This is definitely a bug, please file it in Bugzilla if you wouldn't mind:
> https://bz.apache.org/bugzilla/enter_bug.cgi?product=Tomcat%208
>

For what it's worth, that analysis & approach to fixing seem
reasonable to me.  Yes I'll be glad to file a bug report and test a
patch.

Thanks,
Jason

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Christopher Schultz]

Jason,

On 3/8/16 8:44 PM, Jason Overland wrote:
> Okay, so I checked out Tomcat 8.0.32 from source control. I then
> reverted MemoryRealm's authenticate method to how it was in 7.0.26 and
> built Tomcat and now my authentication works.  This of course is not a
> solution, but it obviates most of my other questions.  I guess the
> important question is: how do I set the CredentialHandler on the
> MemoryRealm?

The way you are setting the CredentialHandler is correct:

>>   
>> > className="org.apache.catalina.realm.MessageDigestCredentialHandler"/>
>>   

You actually don't need to specify the CredentialHandler if you specify
the "digest" attribute, since Tomcat will synthesize a
MessageDigestCredentialHandler for you and use the "digest" as the
"algorithm". But it doesn't hurt to do what you have done.

> For authentication our configuration is using a MemoryRealm with
> digest="SHA".  We are storing usernames and passwords in a
> tomcat-users.xml file.  We are using a jaas.config which specifies to
> use a org.apache.catalina.realm.JAASMemoryLoginModule.  We have our
> own implementation of a CallbackModule.

I must admit I'm not sure how the JAAS configuration fits into all this
(I simply have no JAAS experience). But I suspect that since
JAASMemoryLoginRealm extends RealmBase, it needs to be configured similarly.

The  you have in server.xml looks to be configured correctly, but
I'm not sure it's being used if JAAS is in play. I wouldn't be surprised
if JAAS is creating a separate instance of the JAASMemoryLoginRealm
(which is a MemoryRealm) and never setting any of those properties.

It looks like the best way to set those properties is via the "options"
for the realm:

>> jaas.config:
>> /** JAAS Login Configuration for the Application **/
>>
>> JAASTomcat {
>>org.apache.catalina.realm.JAASMemoryLoginModule required debug=true;
>> };

Instead of simply "debug=true" for the options, we might want to add
"digest=SHA" and then handle it in the initializer for JAASMemoryLoginRealm.

I think the reason that this works in the older code is that the default
algorithm of SHA is built-into the MemoryRealm and doesn't require the
new CredentialHandler stuff. The new method for creating a
CredentialHandler without actually specifying one (i.e. digest="SHA")
requires that the "digest" actually be set to something. Since that's
not happening, you get an NPE.

Since you are willing to build your Tomcat from source, can I give you a
patch to test?

This is definitely a bug, please file it in Bugzilla if you wouldn't mind:
https://bz.apache.org/bugzilla/enter_bug.cgi?product=Tomcat%208

-chris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Jason Overland]

Okay, so I checked out Tomcat 8.0.32 from source control. I then
reverted MemoryRealm's authenticate method to how it was in 7.0.26 and
built Tomcat and now my authentication works.  This of course is not a
solution, but it obviates most of my other questions.  I guess the
important question is: how do I set the CredentialHandler on the
MemoryRealm?

===
--- MemoryRealm.java (revision 1734183)
+++ MemoryRealm.java (working copy)
@@ -115,16 +115,16 @@

 GenericPrincipal principal = principals.get(username);

-boolean validated;
-if (principal == null) {
-validated = false;
-} else {
-if (credentials == null || principal.getPassword() == null) {
-if (log.isDebugEnabled())
-
log.debug(sm.getString("memoryRealm.authenticateFailure", username));
-return (null);
+boolean validated = false;
+if (principal != null && credentials != null) {
+if (hasMessageDigest()) {
+// Hex hashes should be compared case-insensitive
+validated = (digest(credentials)
+ .equalsIgnoreCase(principal.getPassword()));
+} else {
+validated =
+(digest(credentials).equals(principal.getPassword()));
 }
-validated = getCredentialHandler().matches(credentials,
principal.getPassword());
 }

 if (validated) {

On Tue, Mar 8, 2016 at 3:43 PM, Jason Overland  wrote:
> Hi, I'm upgrading an existing Vaadin 6 application from Tomcat 7.0.26
> to Tomcat 8.0.32 and have some questions.  I'm using Windows 7 and
> debugging in Eclipse.
>
> For authentication our configuration is using a MemoryRealm with
> digest="SHA".  We are storing usernames and passwords in a
> tomcat-users.xml file.  We are using a jaas.config which specifies to
> use a org.apache.catalina.realm.JAASMemoryLoginModule.  We have our
> own implementation of a CallbackModule.  For the upgrade I have added
> a CredentialHandler node inside our Realm as suggested in the
> documentation.
>
> server.xml:
>
>   
> 
>  factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
> name="UserDatabase" pathname="conf/tomcat-users.xml"
> type="org.apache.catalina.UserDatabase"/>
>   
>
>   ...
>
>   
>  className="org.apache.catalina.realm.MessageDigestCredentialHandler"/>
>   
>
>
> jaas.config:
> /** JAAS Login Configuration for the Application **/
>
> JAASTomcat {
>org.apache.catalina.realm.JAASMemoryLoginModule required debug=true;
> };
>
> our LoginController:
>
> callbackHandler = new LoginFormCallBackHandler(user);
> tomcatLoginContext = new LoginContext("JAASTomcat", callbackHandler);
> ...
> tomcatLoginContext.login();
>
>
> our CallbackHandler:
> public class LoginFormCallBackHandler implements CallbackHandler,
> Serializable  {
>
> private static final long serialVersionUID = 1L;
> private LoginEvent logins = null;
> private User user;
> public LoginFormCallBackHandler(User user)
> {
> this.user = user;
> }
>
> public void setLogins(LoginEvent loginEvent) {
> logins = loginEvent;
> }
>
> public void handle(Callback[] callbacks) throws IOException,
> UnsupportedCallbackException {
> boolean gotName = false;
> for (int i = 0; i < callbacks.length; i++) {
> if (callbacks[i] instanceof NameCallback) {
> // get the username from the loginform event
> NameCallback nc = (NameCallback) callbacks[i];
>
> nc.setName(logins.getLoginParameter(LoginView.LOGINFORM_PARAMETER_USERNAME));
> user.username = nc.getName();
> gotName = true;
> } else if (callbacks[i] instanceof TextInputCallback) {
> TextInputCallback cb = (TextInputCallback) callbacks[i];
> if (cb.getPrompt().equals("catalinaBase")) {
> cb.setText(System.getenv("CATALINA_BASE"););
> }
> } else if (callbacks[i] instanceof PasswordCallback) {
> // get the password from the loginform event
> PasswordCallback pc = (PasswordCallback) callbacks[i];
> try {
> MessageDigest md = MessageDigest.getInstance("SHA");
>
> md.update(logins.getLoginParameter(LoginView.LOGINFORM_PARAMETER_PASSWORD).getBytes());
> user.password = String.format("%x", new
> BigInteger(1, md.digest()));
> }
> catch (NoSuchAlgorithmException nsae)
> {
> user.password =
> logins.getLoginParameter(LoginView.LOGINFORM_PARAMETER_PASSWORD);
> }
> pc.setPassword(user.password.toCharArray());
> } else {
> System.out.println("Unrecognized Callback: " + callbacks[i]);
> }
> 

NullPointerException in MemoryRealm after upgrading to Tomcat 8.0.32 from 7.0.26

[Jason Overland]

Hi, I'm upgrading an existing Vaadin 6 application from Tomcat 7.0.26
to Tomcat 8.0.32 and have some questions.  I'm using Windows 7 and
debugging in Eclipse.

For authentication our configuration is using a MemoryRealm with
digest="SHA".  We are storing usernames and passwords in a
tomcat-users.xml file.  We are using a jaas.config which specifies to
use a org.apache.catalina.realm.JAASMemoryLoginModule.  We have our
own implementation of a CallbackModule.  For the upgrade I have added
a CredentialHandler node inside our Realm as suggested in the
documentation.

server.xml:

  


  

  ...

  

  


jaas.config:
/** JAAS Login Configuration for the Application **/

JAASTomcat {
   org.apache.catalina.realm.JAASMemoryLoginModule required debug=true;
};

our LoginController:

callbackHandler = new LoginFormCallBackHandler(user);
tomcatLoginContext = new LoginContext("JAASTomcat", callbackHandler);
...
tomcatLoginContext.login();


our CallbackHandler:
public class LoginFormCallBackHandler implements CallbackHandler,
Serializable  {

private static final long serialVersionUID = 1L;
private LoginEvent logins = null;
private User user;
public LoginFormCallBackHandler(User user)
{
this.user = user;
}

public void setLogins(LoginEvent loginEvent) {
logins = loginEvent;
}

public void handle(Callback[] callbacks) throws IOException,
UnsupportedCallbackException {
boolean gotName = false;
for (int i = 0; i < callbacks.length; i++) {
if (callbacks[i] instanceof NameCallback) {
// get the username from the loginform event
NameCallback nc = (NameCallback) callbacks[i];

nc.setName(logins.getLoginParameter(LoginView.LOGINFORM_PARAMETER_USERNAME));
user.username = nc.getName();
gotName = true;
} else if (callbacks[i] instanceof TextInputCallback) {
TextInputCallback cb = (TextInputCallback) callbacks[i];
if (cb.getPrompt().equals("catalinaBase")) {
cb.setText(System.getenv("CATALINA_BASE"););
}
} else if (callbacks[i] instanceof PasswordCallback) {
// get the password from the loginform event
PasswordCallback pc = (PasswordCallback) callbacks[i];
try {
MessageDigest md = MessageDigest.getInstance("SHA");

md.update(logins.getLoginParameter(LoginView.LOGINFORM_PARAMETER_PASSWORD).getBytes());
user.password = String.format("%x", new
BigInteger(1, md.digest()));
}
catch (NoSuchAlgorithmException nsae)
{
user.password =
logins.getLoginParameter(LoginView.LOGINFORM_PARAMETER_PASSWORD);
}
pc.setPassword(user.password.toCharArray());
} else {
System.out.println("Unrecognized Callback: " + callbacks[i]);
}
}
if (gotName)
logins = null;
}
}

This all worked in Tomcat 7 but unfortunately isn't working in Tomcat
8.  I was getting "WARNING: Unable to determine Catalina base to load
file conf/tomcat-users.xml" and so I had to start responding to
TextInputCallback "catalinaBase" as required by the new
getCatalinaBase method on JAASMemoryLoginModule.

I got past that and now I'm getting a NullPointerException:
Authentication failed: java.lang.NullPointerException
at org.apache.catalina.realm.MemoryRealm.authenticate(MemoryRealm.java:127)
at 
org.apache.catalina.realm.JAASMemoryLoginModule.login(JAASMemoryLoginModule.java:288)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at javax.security.auth.login.LoginContext.invoke(Unknown Source)
at javax.security.auth.login.LoginContext.access$000(Unknown Source)
at javax.security.auth.login.LoginContext$4.run(Unknown Source)
at javax.security.auth.login.LoginContext$4.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
at javax.security.auth.login.LoginContext.login(Unknown Source)
at com.lizardtech.es.adminui.vaadin.jaas.LoginController.onLogin(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.vaadin.event.ListenerMethod.receiveEvent(ListenerMethod.java:510)
at com.vaadin.event.EventRouter.fireEvent(EventRouter.java:164)
at com.vaadin.ui.AbstractComponent.fireEvent(AbstractComponent.java:1219)
at com.vaadin.ui.LoginForm$2.handleParameters(LoginForm.java:103)
at