Re: Testing SRV 4.7 (SSL Attributes)

[Ryan Lubke]

section, but optional does not equal do not test.
> 
> I'm going to setup a feedback alias as this input is valuable.  I'll be
> sure to post it once it's all set.  Please *do* send the above to this
> alias once it's ready.
> 
The alias is [EMAIL PROTECTED]  


--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Testing SRV 4.7 (SSL Attributes)

[Ryan Lubke]

On Wed, 2002-09-25 at 01:05, Steve Downey wrote:
> Comments inline ...
> On Tuesday 24 September 2002 11:21 pm, Ryan Lubke wrote:
> > Hi Jason,
> >
> > Comments inline
> >
> > Jason Hunter wrote:
> > >Thanks for the fast response, Ryan.
> > >
> > >Here's how I read the spec.  Simple servlet containers (outside J2EE)
> > >aren't required to support SSL/HTTPS.  However *if a container does
> > >choose to support SSL/HTTPS* then they're required to expose those
> > >attributes.
> > >
> > >Per SRV.4.7: "If a request has been transmitted over a secure protocol,
> > >such as HTTPS, this information must be exposed via the isSecure method
> > >of the ServletRequest interface. The web container must expose the
> > >following attributes to the servlet programmer: ..."
> > >
> > >So the rule for implementors is either (a) don't do SSL or (b) do it
> > >correctly.  I don't think it's appropriate to assume that since
> > >something is optional then we shouldn't test that when present the item
> > >behaves as it should
> >
> > I don't disagree that when using SSL, there are strict requirements that
> > must be followed, but as far as the TCK goes the functionality is
> > optional per the spec and won't be tested (unless there is a shift in
> > testing philosophy).
> >
> 
> It sounds like there is a desparate need to change the testing philosophy. As 
> it stands a container that provides a non-conforming implementation of an 
> optional feature is incorrectly labled as conforming. That's bad.
> 
> If a container does not support an optional section of a test, it's a PASS for 
> the section, but optional does not equal do not test.

I'm going to setup a feedback alias as this input is valuable.  I'll be
sure to post it once it's all set.  Please *do* send the above to this
alias once it's ready.

> 
> > >.
> > >
> > >In fact, I think it's entirely within Watchdog's (and the TCK's) current
> > >goals to ensure that the spec is followed.  Otherwise anything optional
> > >in a JSR is unlikely to work when present.
> >
> > I think it would be easier for Watchdog at this point in time as it's
> > not restricted by
> > being an official TCK.
> >
> 
> So it can be worked on, and the point of including it in the TCK can be argued 
> later.
> 
> > >Here's pseudocode:
> > >
> > >1) Try to connect using HTTPS
> > >2) If fails, return OK
> > >3) If succeeds, check attributes
> > >4) If attributes correct, return OK
> > >5) If attributes incorrect, return FAIL
> > >
> > >In reality the it may need to be the docs that say, "If you support
> > >HTTPS then run this test to make sure you support it fully" in order to
> > >satisfy lines 1 and 2.
> >
> > I agree that it should be documented.  I think if the user is making a
> > conscience
> > decision to test SSL, then step 2 should probably be "If fails, return
> > FAIL".
> >
> 
> I think Jason has the right approach. Check if SSL is present. If it's not, 
> the section PASSes. Actually, it should be UNSUPPORTED, which is a pass for 
> an optional feature. 

My point was not about detection of SSL.  If SSL is being tested, and
the test cannot connect to the container via SSL, then it should not
be considered PASS.  That would be a false positive.


> 
> > >Thoughts, Ryan?
> > >
> > >-jh-
> 
> 
> --
> To unsubscribe, e-mail:   
> For additional commands, e-mail: 
> 



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

[Fwd: Re: Testing SRV 4.7 (SSL Attributes)]

[Ryan Lubke]

> Comments inline ...
> On Tuesday 24 September 2002 11:21 pm, Ryan Lubke wrote:
> > Hi Jason,
> >
> > Comments inline
> >
> > Jason Hunter wrote:
> > >Thanks for the fast response, Ryan.
> > >
> > >Here's how I read the spec.  Simple servlet containers (outside J2EE)
> > >aren't required to support SSL/HTTPS.  However *if a container does
> > >choose to support SSL/HTTPS* then they're required to expose those
> > >attributes.
> > >
> > >Per SRV.4.7: "If a request has been transmitted over a secure protocol,
> > >such as HTTPS, this information must be exposed via the isSecure method
> > >of the ServletRequest interface. The web container must expose the
> > >following attributes to the servlet programmer: ..."
> > >
> > >So the rule for implementors is either (a) don't do SSL or (b) do it
> > >correctly.  I don't think it's appropriate to assume that since
> > >something is optional then we shouldn't test that when present the item
> > >behaves as it should
> >
> > I don't disagree that when using SSL, there are strict requirements that
> > must be followed, but as far as the TCK goes the functionality is
> > optional per the spec and won't be tested (unless there is a shift in
> > testing philosophy).
> >
> 
> It sounds like there is a desparate need to change the testing philosophy. As 
> it stands a container that provides a non-conforming implementation of an 
> optional feature is incorrectly labled as conforming. That's bad.
> 
> If a container does not support an optional section of a test, it's a PASS for 
> the section, but optional does not equal do not test.

I'm going to setup a feedback alias as this input is valuable.  I'll be
sure to post it once it's all set.  Please *do* send the above to this
alias once it's ready.

> 
> > >.
> > >
> > >In fact, I think it's entirely within Watchdog's (and the TCK's) current
> > >goals to ensure that the spec is followed.  Otherwise anything optional
> > >in a JSR is unlikely to work when present.
> >
> > I think it would be easier for Watchdog at this point in time as it's
> > not restricted by
> > being an official TCK.
> >
> 
> So it can be worked on, and the point of including it in the TCK can be argued 
> later.
> 
> > >Here's pseudocode:
> > >
> > >1) Try to connect using HTTPS
> > >2) If fails, return OK
> > >3) If succeeds, check attributes
> > >4) If attributes correct, return OK
> > >5) If attributes incorrect, return FAIL
> > >
> > >In reality the it may need to be the docs that say, "If you support
> > >HTTPS then run this test to make sure you support it fully" in order to
> > >satisfy lines 1 and 2.
> >
> > I agree that it should be documented.  I think if the user is making a
> > conscience
> > decision to test SSL, then step 2 should probably be "If fails, return
> > FAIL".
> >
> 
> I think Jason has the right approach. Check if SSL is present. If it's not, 
> the section PASSes. Actually, it should be UNSUPPORTED, which is a pass for 
> an optional feature. 

My point was not about detection of SSL.  If SSL is being tested, and
the test cannot connect to the container via SSL, then it should not
be considered PASS.  That would be a false positive.


> 
> > >Thoughts, Ryan?
> > >
> > >-jh-
> 
> 
> --
> To unsubscribe, e-mail:   
> For additional commands, e-mail: 
> 




--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Testing SRV 4.7 (SSL Attributes)

[Steve Downey]

Comments inline ...
On Tuesday 24 September 2002 11:21 pm, Ryan Lubke wrote:
> Hi Jason,
>
> Comments inline
>
> Jason Hunter wrote:
> >Thanks for the fast response, Ryan.
> >
> >Here's how I read the spec.  Simple servlet containers (outside J2EE)
> >aren't required to support SSL/HTTPS.  However *if a container does
> >choose to support SSL/HTTPS* then they're required to expose those
> >attributes.
> >
> >Per SRV.4.7: "If a request has been transmitted over a secure protocol,
> >such as HTTPS, this information must be exposed via the isSecure method
> >of the ServletRequest interface. The web container must expose the
> >following attributes to the servlet programmer: ..."
> >
> >So the rule for implementors is either (a) don't do SSL or (b) do it
> >correctly.  I don't think it's appropriate to assume that since
> >something is optional then we shouldn't test that when present the item
> >behaves as it should
>
> I don't disagree that when using SSL, there are strict requirements that
> must be followed, but as far as the TCK goes the functionality is
> optional per the spec and won't be tested (unless there is a shift in
> testing philosophy).
>

It sounds like there is a desparate need to change the testing philosophy. As 
it stands a container that provides a non-conforming implementation of an 
optional feature is incorrectly labled as conforming. That's bad.

If a container does not support an optional section of a test, it's a PASS for 
the section, but optional does not equal do not test.

> >.
> >
> >In fact, I think it's entirely within Watchdog's (and the TCK's) current
> >goals to ensure that the spec is followed.  Otherwise anything optional
> >in a JSR is unlikely to work when present.
>
> I think it would be easier for Watchdog at this point in time as it's
> not restricted by
> being an official TCK.
>

So it can be worked on, and the point of including it in the TCK can be argued 
later.

> >Here's pseudocode:
> >
> >1) Try to connect using HTTPS
> >2) If fails, return OK
> >3) If succeeds, check attributes
> >4) If attributes correct, return OK
> >5) If attributes incorrect, return FAIL
> >
> >In reality the it may need to be the docs that say, "If you support
> >HTTPS then run this test to make sure you support it fully" in order to
> >satisfy lines 1 and 2.
>
> I agree that it should be documented.  I think if the user is making a
> conscience
> decision to test SSL, then step 2 should probably be "If fails, return
> FAIL".
>

I think Jason has the right approach. Check if SSL is present. If it's not, 
the section PASSes. Actually, it should be UNSUPPORTED, which is a pass for 
an optional feature. 

> >Thoughts, Ryan?
> >
> >-jh-


--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Testing SRV 4.7 (SSL Attributes)

[Ryan Lubke]

Hi Jason,

Comments inline

Jason Hunter wrote:

>Thanks for the fast response, Ryan.
>
>Here's how I read the spec.  Simple servlet containers (outside J2EE)
>aren't required to support SSL/HTTPS.  However *if a container does
>choose to support SSL/HTTPS* then they're required to expose those
>attributes.  
>
>Per SRV.4.7: "If a request has been transmitted over a secure protocol,
>such as HTTPS, this information must be exposed via the isSecure method
>of the ServletRequest interface. The web container must expose the
>following attributes to the servlet programmer: ..."
>
>So the rule for implementors is either (a) don't do SSL or (b) do it
>correctly.  I don't think it's appropriate to assume that since
>something is optional then we shouldn't test that when present the item
>behaves as it should
>

I don't disagree that when using SSL, there are strict requirements that 
must be followed, but as far as the TCK goes the functionality is 
optional per the spec and won't be tested (unless there is a shift in 
testing philosophy).

>.  
>
>In fact, I think it's entirely within Watchdog's (and the TCK's) current
>goals to ensure that the spec is followed.  Otherwise anything optional
>in a JSR is unlikely to work when present.
>
I think it would be easier for Watchdog at this point in time as it's 
not restricted by
being an official TCK.

>
>Here's pseudocode:
>
>1) Try to connect using HTTPS
>2) If fails, return OK
>3) If succeeds, check attributes
>4) If attributes correct, return OK
>5) If attributes incorrect, return FAIL
>
>In reality the it may need to be the docs that say, "If you support
>HTTPS then run this test to make sure you support it fully" in order to
>satisfy lines 1 and 2.
>
I agree that it should be documented.  I think if the user is making a 
conscience
decision to test SSL, then step 2 should probably be "If fails, return 
FAIL".

>
>Thoughts, Ryan?
>
>-jh-
>
>Ryan Lubke wrote:
>  
>
>>Jason,
>>
>>The official TCK (and Watchdog) do not perform any SSL testing as this
>>is not a requirement of Servlet containers unless they are a part of a
>>J2EE environment.  Please reference the following sections in the
>>2.3/2.4 specifications:
>>
>>Servlet 2.3/2.4
>>
>>SRV.1.2
>>SRV.12.5.4
>>
>>Watchdog could potentially add these sort of tests as it's not an
>>official TCK, however, I think that would defeat the original idea
>>behind Watchdog.  Of course that doesn't mean a particular projects
>>goals cannot change.
>>
>>-rl
>>
>>Jason Hunter wrote:
>>
>>
>>
>>>Hi all,
>>>
>>>It's come to my attention that most servlet container vendors totally
>>>ignore the requirements laid out Servlet API 2.3 SRV 4.7.  These
>>>requirements are to expose various attributes of an SSL connection via
>>>the javax.servlet.request.cipher_suite, javax.servlet.request.key_size,
>>>and javax.servlet.request.X509Certificate request attributes.
>>>
>>>My theory is that server vendors don't support this requirement because
>>>Watchdog (and presumably the official TCK) don't actually check it, thus
>>>giving server vendors a false sense of compatibility.  Whether my
>>>theory's true or not, I'm confident that if Watchdog (and thus the
>>>official TCK) started checking this requirement then soon enough all
>>>servlet container vendors would support it.  I think that's pretty
>>>important because banks and such need access to these attributes to
>>>ensure a secure connection.
>>>
>>>To that end, I'd like to get a sense of the thoughts here for if
>>>Watchdog can add these sorts of tests.  I don't actually see any
>>>SSL-based tests happening right now, but perhaps I'm not looking in the
>>>right place.  Was that intentional, because of the difficulty setting up
>>>an SSL server?  Is there another reason not to test for the SSL-related
>>>requirements?  How much work would it be to add SSL-related testing?
>>>I'm happy to help to the extent I have time, but would appreciate
>>>hearing the conventional wisdom surrounding these issues.
>>>
>>>-jh-
>>>
>>>--
>>>To unsubscribe, e-mail:   
>>>For additional commands, e-mail: 
>>>
>>>
>>>
>>>.
>>>
>>>
>>>
>>>  
>>>
>>--
>>To unsubscribe, e-mail:   
>>For additional commands, e-mail: 
>>
>>
>
>--
>To unsubscribe, e-mail:   
>For additional commands, e-mail: 
>
>
>
>
>  
>

Re: Testing SRV 4.7 (SSL Attributes)

[Jason Hunter]

Thanks for the fast response, Ryan.

Here's how I read the spec.  Simple servlet containers (outside J2EE)
aren't required to support SSL/HTTPS.  However *if a container does
choose to support SSL/HTTPS* then they're required to expose those
attributes.  

Per SRV.4.7: "If a request has been transmitted over a secure protocol,
such as HTTPS, this information must be exposed via the isSecure method
of the ServletRequest interface. The web container must expose the
following attributes to the servlet programmer: ..."

So the rule for implementors is either (a) don't do SSL or (b) do it
correctly.  I don't think it's appropriate to assume that since
something is optional then we shouldn't test that when present the item
behaves as it should.  

In fact, I think it's entirely within Watchdog's (and the TCK's) current
goals to ensure that the spec is followed.  Otherwise anything optional
in a JSR is unlikely to work when present.

Here's pseudocode:

1) Try to connect using HTTPS
2) If fails, return OK
3) If succeeds, check attributes
4) If attributes correct, return OK
5) If attributes incorrect, return FAIL

In reality the it may need to be the docs that say, "If you support
HTTPS then run this test to make sure you support it fully" in order to
satisfy lines 1 and 2.

Thoughts, Ryan?

-jh-

Ryan Lubke wrote:
> 
> Jason,
> 
> The official TCK (and Watchdog) do not perform any SSL testing as this
> is not a requirement of Servlet containers unless they are a part of a
> J2EE environment.  Please reference the following sections in the
> 2.3/2.4 specifications:
> 
> Servlet 2.3/2.4
> 
> SRV.1.2
> SRV.12.5.4
> 
> Watchdog could potentially add these sort of tests as it's not an
> official TCK, however, I think that would defeat the original idea
> behind Watchdog.  Of course that doesn't mean a particular projects
> goals cannot change.
> 
> -rl
> 
> Jason Hunter wrote:
> 
> >Hi all,
> >
> >It's come to my attention that most servlet container vendors totally
> >ignore the requirements laid out Servlet API 2.3 SRV 4.7.  These
> >requirements are to expose various attributes of an SSL connection via
> >the javax.servlet.request.cipher_suite, javax.servlet.request.key_size,
> >and javax.servlet.request.X509Certificate request attributes.
> >
> >My theory is that server vendors don't support this requirement because
> >Watchdog (and presumably the official TCK) don't actually check it, thus
> >giving server vendors a false sense of compatibility.  Whether my
> >theory's true or not, I'm confident that if Watchdog (and thus the
> >official TCK) started checking this requirement then soon enough all
> >servlet container vendors would support it.  I think that's pretty
> >important because banks and such need access to these attributes to
> >ensure a secure connection.
> >
> >To that end, I'd like to get a sense of the thoughts here for if
> >Watchdog can add these sorts of tests.  I don't actually see any
> >SSL-based tests happening right now, but perhaps I'm not looking in the
> >right place.  Was that intentional, because of the difficulty setting up
> >an SSL server?  Is there another reason not to test for the SSL-related
> >requirements?  How much work would it be to add SSL-related testing?
> >I'm happy to help to the extent I have time, but would appreciate
> >hearing the conventional wisdom surrounding these issues.
> >
> >-jh-
> >
> >--
> >To unsubscribe, e-mail:   
> >For additional commands, e-mail: 
> >
> >
> >
> >.
> >
> >
> >
> 
> --
> To unsubscribe, e-mail:   
> For additional commands, e-mail: 

--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Testing SRV 4.7 (SSL Attributes)

[Ryan Lubke]

Jason,

The official TCK (and Watchdog) do not perform any SSL testing as this 
is not a requirement of Servlet containers unless they are a part of a 
J2EE environment.  Please reference the following sections in the 
2.3/2.4 specifications:

Servlet 2.3/2.4

SRV.1.2
SRV.12.5.4

Watchdog could potentially add these sort of tests as it's not an 
official TCK, however, I think that would defeat the original idea 
behind Watchdog.  Of course that doesn't mean a particular projects 
goals cannot change.

-rl

Jason Hunter wrote:

>Hi all,
>
>It's come to my attention that most servlet container vendors totally
>ignore the requirements laid out Servlet API 2.3 SRV 4.7.  These
>requirements are to expose various attributes of an SSL connection via
>the javax.servlet.request.cipher_suite, javax.servlet.request.key_size,
>and javax.servlet.request.X509Certificate request attributes.
>
>My theory is that server vendors don't support this requirement because
>Watchdog (and presumably the official TCK) don't actually check it, thus
>giving server vendors a false sense of compatibility.  Whether my
>theory's true or not, I'm confident that if Watchdog (and thus the
>official TCK) started checking this requirement then soon enough all
>servlet container vendors would support it.  I think that's pretty
>important because banks and such need access to these attributes to
>ensure a secure connection.
>
>To that end, I'd like to get a sense of the thoughts here for if
>Watchdog can add these sorts of tests.  I don't actually see any
>SSL-based tests happening right now, but perhaps I'm not looking in the
>right place.  Was that intentional, because of the difficulty setting up
>an SSL server?  Is there another reason not to test for the SSL-related
>requirements?  How much work would it be to add SSL-related testing? 
>I'm happy to help to the extent I have time, but would appreciate
>hearing the conventional wisdom surrounding these issues.
>
>-jh-
>
>--
>To unsubscribe, e-mail:   
>For additional commands, e-mail: 
>
>
>
>.
>
>  
>



--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Testing SRV 4.7 (SSL Attributes)

[Jason Hunter]

Hi all,

It's come to my attention that most servlet container vendors totally
ignore the requirements laid out Servlet API 2.3 SRV 4.7.  These
requirements are to expose various attributes of an SSL connection via
the javax.servlet.request.cipher_suite, javax.servlet.request.key_size,
and javax.servlet.request.X509Certificate request attributes.

My theory is that server vendors don't support this requirement because
Watchdog (and presumably the official TCK) don't actually check it, thus
giving server vendors a false sense of compatibility.  Whether my
theory's true or not, I'm confident that if Watchdog (and thus the
official TCK) started checking this requirement then soon enough all
servlet container vendors would support it.  I think that's pretty
important because banks and such need access to these attributes to
ensure a secure connection.

To that end, I'd like to get a sense of the thoughts here for if
Watchdog can add these sorts of tests.  I don't actually see any
SSL-based tests happening right now, but perhaps I'm not looking in the
right place.  Was that intentional, because of the difficulty setting up
an SSL server?  Is there another reason not to test for the SSL-related
requirements?  How much work would it be to add SSL-related testing? 
I'm happy to help to the extent I have time, but would appreciate
hearing the conventional wisdom surrounding these issues.

-jh-

--
To unsubscribe, e-mail:   
For additional commands, e-mail: