Re: TLS1.3 change_cipher_spec as part of application data
I checked this further and the issue was indeed with my code.. I did some recent async io completion handler refactor because of which the bio's socket write completion was triggering the observer's read completion callback.. the records were actually client side write bio buffers which got mixed with the ssl_read data on the read path. Thanks for the pointer, we are doing good. Nothing further is needed at this point. Neelabh On Thu, Nov 24, 2022 at 4:51 PM Neelabh Mam wrote: > I hook an observer for decrypted data immediately after the handshake is > successful (SSL_do_handshake rc 1) and it is this observer which gets the > ccs+list data on the vert next ssl_read cycle. Now, it could be that my > code is at fault here.. But I do see the decrypted dummy ccs and one more > record as part of the list data. Annotated hex dump of the buffer > reproduced below. Btw this is against the latest FZ windows FTPS server > which I think recently moved to GNUtls. As suggested earlier I'll check > my code again and open a github ticket. > > Command : PASV > sock-cc 2570925053120 OnWrite, n 29 > sock-cc 2570925053120 OnRead, n 273 > *New session callback 1* > sock-cc 2570925053120 OnRead, n 68 > Response : 227 Entering Passive Mode (127,0,0,1,249,75) > Command : LIST / > sock-dc 2570925624800 OnConnect, n 0 > sock-dc OnConnect() > sock-cc 2570925053120 OnWrite, n 30 > sock-cc 2570925053120 OnRead, n 51 > Response : 150 Starting data transfer. > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 32 > sock-dc 2570925624800 OnWrite, n 10 > sock-dc 2570925624800 OnRead, n 133 > sock-dc Handshake failed : -1 2 > sock-dc 2570925624800 OnRead, n 108 > *sock-dc Handshake done. TLS session reused : 1* > sock-dc 2570925624800 OnWrite, n 32 > observer OnDataChannelIoCompletion write > sock-dc 2570925624800 OnWrite, n 32 > observer OnDataChannelIoCompletion write > sock-dc 2570925624800 OnWrite, n 16 > observer OnDataChannelIoCompletion write > sock-dc 2570925624800 OnRead, n 86 > observer OnDataChannelIoCompletion read > sock-cc 2570925053120 OnRead, n 48 > Response : 226 Operation successful > > sock-dc 2570925624800 OnRead, n 816 > sock-dc SSL_RECEIVED_SHUTDOWN, flags : 2 > sock-dc : ssl_shutdown() rc : 1 > sock-dc SSL_RECEIVED_SHUTDOWN, flags : 3 > sock-dc : shutdown(sd_send) > observer OnDataChannelIoCompletion read > sock-dc 2570925624800 OnRead, n 0 > sock-dc shutdown mode : 3 > ¢â╣ª5d²┼l⌐\╒═£tdrwxrwxrwx 1 ftp ftp 0 Nov 01 08:55 > $RECYCLE.BIN > -rw-rw-rw- 1 ftp ftp 1468320 Oct 15 17:37 accesschk.exe > -rw-rw-rw- 1 ftp ftp 810416 Oct 15 17:37 accesschk64.exe > -rw-rw-rw- 1 ftp ftp 264088 Oct 15 17:37 AccessEnum.exe > -rw-rw-rw- 1 ftp ftp 2502032 Oct 15 17:37 Autoruns.exe > -rw-rw-rw- 1 ftp ftp 2928520 Oct 15 17:37 Autoruns64.exe > -rw-rw-rw- 1 ftp ftp 712080 Oct 15 17:37 autorunsc.exe > -rw-rw-rw- 1 ftp ftp 788400 Oct 15 17:37 autorunsc64.exe > drwxrwxrwx 1 ftp ftp 0 Nov 06 16:52 System Volume Information > -rw-rw-rw- 1 ftp ftp 5 Nov 24 02:26 x.txt > > 14 TLS Change Cipher Spec protocol > 03 03 SSL 3.3 (TLS 1.2) > 00 01 length of change cipher spec message > 01 payload (irrelevant) > > 17TLS application data > 03 03 version 1.2 > 00 45 length of payload (69 bytes) > > 14 03 03 00 01 01 *17 03 03 00 45* 4b 60 8c 72 be > 99 60 ed da 3a a0 09 9c 35 24 a5 3d 48 b5 78 bc > 2e 2c c3 e7 19 b4 74 e9 7b 70 90 d4 17 d4 2a 46 > 8c e7 1a 94 6b c8 ea 54 b2 72 8b 03 8e cb 0d 9b > 83 b9 a6 35 64 fd c5 55 08 6c a9 5c d5 cd 9c 74 > 64 72 77 78 72 77 78 72 77 78 20 31 20 66 74 70 > 20 66 74 70 20 20 20 20 20 20 20 20 20 20 20 20 > 20 20 20 30 20 4e 6f 76 20 30 31 20 30 38 3a 35 > 35 20 24 52 45 43 59 43 4c 45 2e 42 49 4e 0d 0a > 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70 > 20 66 74 70 20 20 20 20 20 20 20 20 20 31 34 36 > 38 33 32 30 20 4f 63 74 20 31 35 20 31 37 3a 33 > 37 20 61 63 63 65 73 73 63 68 6b 2e 65 78 65 0d > 0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 > 70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 38 > 31
Re: TLS1.3 change_cipher_spec as part of application data
I hook an observer for decrypted data immediately after the handshake is successful (SSL_do_handshake rc 1) and it is this observer which gets the ccs+list data on the vert next ssl_read cycle. Now, it could be that my code is at fault here.. But I do see the decrypted dummy ccs and one more record as part of the list data. Annotated hex dump of the buffer reproduced below. Btw this is against the latest FZ windows FTPS server which I think recently moved to GNUtls. As suggested earlier I'll check my code again and open a github ticket. Command : PASV sock-cc 2570925053120 OnWrite, n 29 sock-cc 2570925053120 OnRead, n 273 *New session callback 1* sock-cc 2570925053120 OnRead, n 68 Response : 227 Entering Passive Mode (127,0,0,1,249,75) Command : LIST / sock-dc 2570925624800 OnConnect, n 0 sock-dc OnConnect() sock-cc 2570925053120 OnWrite, n 30 sock-cc 2570925053120 OnRead, n 51 Response : 150 Starting data transfer. sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 32 sock-dc 2570925624800 OnWrite, n 10 sock-dc 2570925624800 OnRead, n 133 sock-dc Handshake failed : -1 2 sock-dc 2570925624800 OnRead, n 108 *sock-dc Handshake done. TLS session reused : 1* sock-dc 2570925624800 OnWrite, n 32 observer OnDataChannelIoCompletion write sock-dc 2570925624800 OnWrite, n 32 observer OnDataChannelIoCompletion write sock-dc 2570925624800 OnWrite, n 16 observer OnDataChannelIoCompletion write sock-dc 2570925624800 OnRead, n 86 observer OnDataChannelIoCompletion read sock-cc 2570925053120 OnRead, n 48 Response : 226 Operation successful sock-dc 2570925624800 OnRead, n 816 sock-dc SSL_RECEIVED_SHUTDOWN, flags : 2 sock-dc : ssl_shutdown() rc : 1 sock-dc SSL_RECEIVED_SHUTDOWN, flags : 3 sock-dc : shutdown(sd_send) observer OnDataChannelIoCompletion read sock-dc 2570925624800 OnRead, n 0 sock-dc shutdown mode : 3 ¢â╣ª5d²┼l⌐\╒═£tdrwxrwxrwx 1 ftp ftp 0 Nov 01 08:55 $RECYCLE.BIN -rw-rw-rw- 1 ftp ftp 1468320 Oct 15 17:37 accesschk.exe -rw-rw-rw- 1 ftp ftp 810416 Oct 15 17:37 accesschk64.exe -rw-rw-rw- 1 ftp ftp 264088 Oct 15 17:37 AccessEnum.exe -rw-rw-rw- 1 ftp ftp 2502032 Oct 15 17:37 Autoruns.exe -rw-rw-rw- 1 ftp ftp 2928520 Oct 15 17:37 Autoruns64.exe -rw-rw-rw- 1 ftp ftp 712080 Oct 15 17:37 autorunsc.exe -rw-rw-rw- 1 ftp ftp 788400 Oct 15 17:37 autorunsc64.exe drwxrwxrwx 1 ftp ftp 0 Nov 06 16:52 System Volume Information -rw-rw-rw- 1 ftp ftp 5 Nov 24 02:26 x.txt 14 TLS Change Cipher Spec protocol 03 03 SSL 3.3 (TLS 1.2) 00 01 length of change cipher spec message 01 payload (irrelevant) 17TLS application data 03 03 version 1.2 00 45 length of payload (69 bytes) 14 03 03 00 01 01 *17 03 03 00 45* 4b 60 8c 72 be 99 60 ed da 3a a0 09 9c 35 24 a5 3d 48 b5 78 bc 2e 2c c3 e7 19 b4 74 e9 7b 70 90 d4 17 d4 2a 46 8c e7 1a 94 6b c8 ea 54 b2 72 8b 03 8e cb 0d 9b 83 b9 a6 35 64 fd c5 55 08 6c a9 5c d5 cd 9c 74 64 72 77 78 72 77 78 72 77 78 20 31 20 66 74 70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 30 20 4e 6f 76 20 30 31 20 30 38 3a 35 35 20 24 52 45 43 59 43 4c 45 2e 42 49 4e 0d 0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70 20 66 74 70 20 20 20 20 20 20 20 20 20 31 34 36 38 33 32 30 20 4f 63 74 20 31 35 20 31 37 3a 33 37 20 61 63 63 65 73 73 63 68 6b 2e 65 78 65 0d 0a 2d 72 77 2d 72 77 2d 72 77 2d 20 31 20 66 74 70 20 66 74 70 20 20 20 20 20 20 20 20 20 20 38 31 30 34 31 36 20 4f 63 74 20 31 35 20 31 37 3a 33 37 20 61 63 63 65 73 73 63 68 6b 36 34 2e 65 : : goes on full list On Thu, Nov 24, 2022 at 3:15 PM Matt Caswell wrote: > > > On 24/11/2022 07:57, Neelabh Mam wrote: > > Hi, > > > > With my openssl based FTPS client (non-blocking bio) targeting TLS1.3, I > > see that immediately after a successful data channel handshake (with > > session reuse), a dummy change_cipher_spec record and a non-application > > data record are sent as part of the directory listing data. Same holds > > true for file downloads as well..(again with session reuse). This seems > > to be in line with the last paragraph
Re: TLS1.3 change_cipher_spec as part of application data
On 24/11/2022 07:57, Neelabh Mam wrote: Hi, With my openssl based FTPS client (non-blocking bio) targeting TLS1.3, I see that immediately after a successful data channel handshake (with session reuse), a dummy change_cipher_spec record and a non-application data record are sent as part of the directory listing data. Same holds true for file downloads as well..(again with session reuse). This seems to be in line with the last paragraph in Appendix D.4 of RFC8446 which mandates a change_cipher_spec record to be sent from the server in case of session reuse. I could manually filter out the non-application data records from the actual data.. However, I was wondering if there's some api/setting in openssl that would do that at an API level.. as this looks like it should ideally be part of openssl workflow. But then what I am also not sure about is why the receipt of new session ticket data on the control channel does not bubble up as application data ? Openssl, from what I can tell, seems to be "consuming" it (new session data) internally. The earlier 2 messages however, do get exposed out of ssl_read and eventually become part of application data. The only data you should be getting out of an `SSL_read` call is application data. If you're getting handshake or CCS messages as part of app data then something has gone very surprisingly wrong. I cannot imagine what would cause that. You might want to open a github issue about that to dive into it in more detail. Matt Is it like : the message type warrants one to be exposed (change_cipher_spec) and the other to be handled internally (new session data) ? Could we please advise on openssl's standard operating workflow here ? Also, would I have to add logic to manually separate these 2 non-application records from the application data ? Is that how it is supposed to be ? Thank you. Neelabh
