[ 
https://issues.apache.org/jira/browse/PROTON-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Harvey updated PROTON-171:
---------------------------------

    Description: 
We've been working on the Java SSL implementation and are seeing a test fail 
against proton-c but that works against proton-j.  We believe this is due to a 
bug in proton-c.

One of the scenarios we wanted to cover in our testing was the case where the 
Transport input method leaves "left-overs", e.g. when you call server.input() 
with 100 bytes of input, but it only accepts 20, as indicated by its return 
value.  

For example, we expect this to happen if the preceding client.output() call is 
told to write to a buffer sized such that its output contains a trailing 
*fragment* of an SSL packet, which input() won't be able to decipher.

We therefore modified the pump method in proton/tests/proton_tests/ssl.py to 
handle this case.  In its loop, it now captures the bytes "left over" after 
calling input(), and prepends them to the input() invocation in the next 
iteration.  The buffer size is now a parameter so individual tests can exercise 
the packet fragmenting behaviour described above.

We made the following change:

-------
diff --git a/tests/proton_tests/ssl.py b/tests/proton_tests/ssl.py
index 8567b1b..237c3da 100644
--- a/tests/proton_tests/ssl.py
+++ b/tests/proton_tests/ssl.py
@@ -43,13 +43,32 @@ class SslTest(common.Test):
         self.t_client = None
         self.t_server = None
 
-    def _pump(self):
+    def _pump(self, buffer_size=1024):
+        """
+        Make the transport send up to buffer_size bytes (this will be the AMQP
+        header and open frame) returning a buffer containing the bytes
+        sent.  Transport is stateful so this will return 0 when it has
+        no more frames to send.
+        TODO this function is duplicated in sasl.py. Should be moved to a 
common place.
+        """
+        out_client_leftover_by_server = ""
+        out_server_leftover_by_client = ""
+        i=0
         while True:
-            out_client = self.t_client.output(1024)
-            out_server = self.t_server.output(1024)
-            if out_client: self.t_server.input(out_client)
-            if out_server: self.t_client.input(out_server)
+
+            out_client = out_client_leftover_by_server + 
self.t_client.output(buffer_size)
+            out_server = out_server_leftover_by_client + 
self.t_server.output(buffer_size)
+
+            if out_client:
+                number_server_consumed = self.t_server.input(out_client)
+                out_client_leftover_by_server = 
out_client[number_server_consumed:] # if it consumed everything then this is 
empty
+
+            if out_server:
+                number_client_consumed = self.t_client.input(out_server)
+                out_server_leftover_by_client = 
out_server[number_client_consumed:] # if it consumed everything then this is 
empty
+
             if not out_client and not out_server: break
+            i=i+1
 
     def _testpath(self, file):
         """ Set the full path to the certificate,keyfile, etc. for the test.
-------

Several ssl tests now fail when run against proton-c, all with the same error.  
This surprised us because we hadn't started playing with the buffer size yet - 
we were still using the default of 1024.

For example, test_server_authentication gives this output:

-------
proton_tests.ssl.SslTest.test_server_authentication 
.........................................................................[0xa2ca208:0]
 ERROR[-2] SSL Failure: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption 
failed or bad record mac
 fail
Error during test:  Traceback (most recent call last):
    File "./tests/proton-test", line 331, in run
      phase()
    File "/home/phil/dev/proton/tests/proton_tests/ssl.py", line 166, in 
test_server_authentication
      self._pump()
    File "/home/phil/dev/proton/tests/proton_tests/ssl.py", line 63, in _pump
      number_server_consumed = self.t_server.input(out_client)
    File "/home/phil/dev/proton/proton-c/bindings/python/proton.py", line 2141, 
in input
      return self._check(n)
    File "/home/phil/dev/proton/proton-c/bindings/python/proton.py", line 2115, 
in _check
      raise exc("[%s]: %s" % (err, 
pn_error_text(pn_transport_error(self._trans))))
  TransportException: [-2]: SSL Failure: error:1408F119:SSL 
routines:SSL3_GET_RECORD:decryption failed or bad record mac
Totals: 1 tests, 0 passed, 0 skipped, 0 ignored, 1 failed
-------

The first pump() call in this test works fine; the failure we see is when it's 
invoked again after closing the connection.

The problem is that the previous t_server.input call didn't accept *any* of the 
bytes given to it.  On the next t_server.input call, these bytes are prepended 
to the newly-produced ones from t_client.output, which seems reasonable to us, 
but this produces the error above.


  was:
We've been working on the Java SSL implementation and are seeing a test fail 
against proton-c but that works against proton-j.  We believe this is due to a 
bug in proton-c.

One of the scenarios we wanted to cover in our testing was the case where the 
Transport input method leaves "left-overs", e.g. when you call server.input() 
with 100 bytes of input, but it only accepts 20, as indicated by its return 
value.  

For example, we expect this to happen if the preceding client.output() call is 
told to write to a buffer sized such that its output contains a trailing 
*fragment* of an SSL packet, which input() won't be able to decipher.

We therefore modified the pump method in proton/tests/proton_tests/ssl.py to 
handle this case.  In its loop, it now captures the bytes "left over" after 
calling input(), and prepends them to the input() invocation in the next 
iteration.  The buffer size is now a parameter so individual tests can exercise 
the packet fragmenting behaviour described above.

We made the following change:

{noformat}
diff --git a/tests/proton_tests/ssl.py b/tests/proton_tests/ssl.py
index 8567b1b..237c3da 100644
--- a/tests/proton_tests/ssl.py
+++ b/tests/proton_tests/ssl.py
@@ -43,13 +43,32 @@ class SslTest(common.Test):
         self.t_client = None
         self.t_server = None
 
-    def _pump(self):
+    def _pump(self, buffer_size=1024):
+        """
+        Make the transport send up to buffer_size bytes (this will be the AMQP
+        header and open frame) returning a buffer containing the bytes
+        sent.  Transport is stateful so this will return 0 when it has
+        no more frames to send.
+        TODO this function is duplicated in sasl.py. Should be moved to a 
common place.
+        """
+        out_client_leftover_by_server = ""
+        out_server_leftover_by_client = ""
+        i=0
         while True:
-            out_client = self.t_client.output(1024)
-            out_server = self.t_server.output(1024)
-            if out_client: self.t_server.input(out_client)
-            if out_server: self.t_client.input(out_server)
+
+            out_client = out_client_leftover_by_server + 
self.t_client.output(buffer_size)
+            out_server = out_server_leftover_by_client + 
self.t_server.output(buffer_size)
+
+            if out_client:
+                number_server_consumed = self.t_server.input(out_client)
+                out_client_leftover_by_server = 
out_client[number_server_consumed:] # if it consumed everything then this is 
empty
+
+            if out_server:
+                number_client_consumed = self.t_client.input(out_server)
+                out_server_leftover_by_client = 
out_server[number_client_consumed:] # if it consumed everything then this is 
empty
+
             if not out_client and not out_server: break
+            i=i+1
 
     def _testpath(self, file):
         """ Set the full path to the certificate,keyfile, etc. for the test.
{noformat}

Several ssl tests now fail when run against proton-c, all with the same error.  
This surprised us because we hadn't started playing with the buffer size yet - 
we were still using the default of 1024.

For example, test_server_authentication gives this output:

{noformat}
proton_tests.ssl.SslTest.test_server_authentication 
.........................................................................[0xa2ca208:0]
 ERROR[-2] SSL Failure: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption 
failed or bad record mac
 fail
Error during test:  Traceback (most recent call last):
    File "./tests/proton-test", line 331, in run
      phase()
    File "/home/phil/dev/proton/tests/proton_tests/ssl.py", line 166, in 
test_server_authentication
      self._pump()
    File "/home/phil/dev/proton/tests/proton_tests/ssl.py", line 63, in _pump
      number_server_consumed = self.t_server.input(out_client)
    File "/home/phil/dev/proton/proton-c/bindings/python/proton.py", line 2141, 
in input
      return self._check(n)
    File "/home/phil/dev/proton/proton-c/bindings/python/proton.py", line 2115, 
in _check
      raise exc("[%s]: %s" % (err, 
pn_error_text(pn_transport_error(self._trans))))
  TransportException: [-2]: SSL Failure: error:1408F119:SSL 
routines:SSL3_GET_RECORD:decryption failed or bad record mac
Totals: 1 tests, 0 passed, 0 skipped, 0 ignored, 1 failed
{noformat}

The first pump() call in this test works fine; the failure we see is when it's 
invoked again after closing the connection.

The problem is that the previous t_server.input call didn't accept *any* of the 
bytes given to it.  On the next t_server.input call, these bytes are prepended 
to the newly-produced ones from t_client.output, which seems reasonable to us, 
but this produces the error above.


    
> after connection close, proton-c SSL decryption fails when left-over bytes 
> passed to input()
> --------------------------------------------------------------------------------------------
>
>                 Key: PROTON-171
>                 URL: https://issues.apache.org/jira/browse/PROTON-171
>             Project: Qpid Proton
>          Issue Type: Bug
>          Components: proton-c
>            Reporter: Philip Harvey
>
> We've been working on the Java SSL implementation and are seeing a test fail 
> against proton-c but that works against proton-j.  We believe this is due to 
> a bug in proton-c.
> One of the scenarios we wanted to cover in our testing was the case where the 
> Transport input method leaves "left-overs", e.g. when you call server.input() 
> with 100 bytes of input, but it only accepts 20, as indicated by its return 
> value.  
> For example, we expect this to happen if the preceding client.output() call 
> is told to write to a buffer sized such that its output contains a trailing 
> *fragment* of an SSL packet, which input() won't be able to decipher.
> We therefore modified the pump method in proton/tests/proton_tests/ssl.py to 
> handle this case.  In its loop, it now captures the bytes "left over" after 
> calling input(), and prepends them to the input() invocation in the next 
> iteration.  The buffer size is now a parameter so individual tests can 
> exercise the packet fragmenting behaviour described above.
> We made the following change:
> -------
> diff --git a/tests/proton_tests/ssl.py b/tests/proton_tests/ssl.py
> index 8567b1b..237c3da 100644
> --- a/tests/proton_tests/ssl.py
> +++ b/tests/proton_tests/ssl.py
> @@ -43,13 +43,32 @@ class SslTest(common.Test):
>          self.t_client = None
>          self.t_server = None
>  
> -    def _pump(self):
> +    def _pump(self, buffer_size=1024):
> +        """
> +        Make the transport send up to buffer_size bytes (this will be the 
> AMQP
> +        header and open frame) returning a buffer containing the bytes
> +        sent.  Transport is stateful so this will return 0 when it has
> +        no more frames to send.
> +        TODO this function is duplicated in sasl.py. Should be moved to a 
> common place.
> +        """
> +        out_client_leftover_by_server = ""
> +        out_server_leftover_by_client = ""
> +        i=0
>          while True:
> -            out_client = self.t_client.output(1024)
> -            out_server = self.t_server.output(1024)
> -            if out_client: self.t_server.input(out_client)
> -            if out_server: self.t_client.input(out_server)
> +
> +            out_client = out_client_leftover_by_server + 
> self.t_client.output(buffer_size)
> +            out_server = out_server_leftover_by_client + 
> self.t_server.output(buffer_size)
> +
> +            if out_client:
> +                number_server_consumed = self.t_server.input(out_client)
> +                out_client_leftover_by_server = 
> out_client[number_server_consumed:] # if it consumed everything then this is 
> empty
> +
> +            if out_server:
> +                number_client_consumed = self.t_client.input(out_server)
> +                out_server_leftover_by_client = 
> out_server[number_client_consumed:] # if it consumed everything then this is 
> empty
> +
>              if not out_client and not out_server: break
> +            i=i+1
>  
>      def _testpath(self, file):
>          """ Set the full path to the certificate,keyfile, etc. for the test.
> -------
> Several ssl tests now fail when run against proton-c, all with the same 
> error.  This surprised us because we hadn't started playing with the buffer 
> size yet - we were still using the default of 1024.
> For example, test_server_authentication gives this output:
> -------
> proton_tests.ssl.SslTest.test_server_authentication 
> .........................................................................[0xa2ca208:0]
>  ERROR[-2] SSL Failure: error:1408F119:SSL 
> routines:SSL3_GET_RECORD:decryption failed or bad record mac
>  fail
> Error during test:  Traceback (most recent call last):
>     File "./tests/proton-test", line 331, in run
>       phase()
>     File "/home/phil/dev/proton/tests/proton_tests/ssl.py", line 166, in 
> test_server_authentication
>       self._pump()
>     File "/home/phil/dev/proton/tests/proton_tests/ssl.py", line 63, in _pump
>       number_server_consumed = self.t_server.input(out_client)
>     File "/home/phil/dev/proton/proton-c/bindings/python/proton.py", line 
> 2141, in input
>       return self._check(n)
>     File "/home/phil/dev/proton/proton-c/bindings/python/proton.py", line 
> 2115, in _check
>       raise exc("[%s]: %s" % (err, 
> pn_error_text(pn_transport_error(self._trans))))
>   TransportException: [-2]: SSL Failure: error:1408F119:SSL 
> routines:SSL3_GET_RECORD:decryption failed or bad record mac
> Totals: 1 tests, 0 passed, 0 skipped, 0 ignored, 1 failed
> -------
> The first pump() call in this test works fine; the failure we see is when 
> it's invoked again after closing the connection.
> The problem is that the previous t_server.input call didn't accept *any* of 
> the bytes given to it.  On the next t_server.input call, these bytes are 
> prepended to the newly-produced ones from t_client.output, which seems 
> reasonable to us, but this produces the error above.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Reply via email to