Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[Cyril Soler]

I suggest that you use your SSH tunnel manually: send clear packets with some 
ID in them (packet
number, whatever you prefer)
and make sure that you get them all on the other end of the tunnel, without 
major delay. Normally
this should work,
but if it does not, and if you're able to create a working prototype that shows 
the bug, I can take
a look at libretroshare to find the problem for you.

Otherwise, because problems occur mainly when X is used, it might be that X 
expects some particular
params of the SSH connection (which you get probably when doing "ssh -X or ssh 
-Y" but not using
your own tunnel system). It's probably worth looking into this.

On 28/07/2016 10:04, jeetu.gol...@gmail.com wrote:
> Hello Cyril,
>
> To report back, I tried out the latest RS master codebase from github
> with my plugin as per your suggestion. Unfortunately the issue is
> still the same.
>
> While I said earlier most of the times (say 90%) the failure occurs
> when there are many small bytes of approx 172 bytes following by
> something like 7556 bytes or some such number (usually above 3000
> bytes).
>
> However, there are instances when the freeze-up occurs with a
> relatively short number of bytesmaybe as low as 60 or 70 bytes. So
> when I tried running gedit, during startup itself, the ssh client end
> froze. While most of the time I will get the main interface window to
> the X client and it freezes somewhere along the line, there are also
> moments such as these when it just hands early on itself.
>
> I also tried to log in to just a SSH console session via the RS tunnel
> plugin and did a "du" on my root directory just to generate a lot of
> text traffic in the form of file names scrolling as du goes through my
> directory. This worked fine and didn't baulk. However this situation
> is a little different from a SSH X forward since here there is only
> unidrectional traffic (from ssh server to ssh client). Whereas with a
> SSH forward there is a lot of bi-directional traffic. Dunno if that
> helps, just trying to get as many clues as possible,
>
> I am also not using any message queue or buffer within my own code, so
> as information comes in from ssh client or server I push it down the
> pipe via sendItem. Not sure if somehow the information coming in from
> these ends is going out of sync and causing a freeze-up.
>
> Your suggestion of this being related to the bug you had closed
> wherein the SSL stream would lock up on occasion waiting for another
> packed, definitely seemed to fit the profile of the above behaviour.
>
> Is there some other way we can capture additional data from within RS
> that would help you confirm or rule out if this problem still persists
> or that would help us concretely ascertain if my code is messing up
> things or not?
>
> Since the SSH traffic is all encrypted, I am unable to verify if the
> data itself gets corrupted somewhere along the line. Though if that
> were to happen the SSH client / server should cut off connection with
> a bad MAC / bad HMAC / Bad Packet Length or some such message if I
> understand it correctly. Though to rule this out I am considering
> building a python client and server that will try to push through
> controlled data to and fro this pipe, and try to replicate this
> freezehopefully something there will then be able to clarify if
> data corruption is occurring and is the problem.
>
> As you can say I am fast approaching a state where I am desperate for
> ideas ;).any help or suggestion would help my sanity to a great
> deal :)
>
> Thank you so much for your time and help :)
>
> Bye for now.
>
>
>
> On Mon, Jul 25, 2016 at 11:46 PM, jeetu.gol...@gmail.com
>  wrote:
>> Hi Cyril,
>>
>> Thank you so much for taking the time to reply :)
>>
>>> This does not seem like a crazy amount of traffic either. I'm not sure 
>>> which >version of RS you're
>> I've been working with RS v0.6.0
>>
>>> using, but a few weeks ago, I fixed a bug that would cause SSL packets to 
>>> >hang in the pipeline until
>>> a new packet arrives. This could be the reason as your X server is maybe 
>>> >waiting for the stream to
>>> reach a given point before continuing.
>> Oh wow :) so there is hope :).I shall download the latest master
>> of RS and try again with thatwill keep you posted on the progress
>> I make :)
>>
>>> As I understand, you developed a RS service that streams SSL connection 
>>> >binary items through RS
>>> friend connections?
>>> You're most probably using RsItem objects then. Their size is limited to 
>>> >262144 bytes. Any packet
>>> larger gets dropped.
>> Yes I have been using RsItem objects...and this explains so much
>> :).is there some other class object I should use wherein this
>> limit is not there / or more relaxed? Though at this moment this isn't
>> a problem for me since I chunk the data and then send it. However it
>> would be good to know what my options here are :)
>>
>>
>> This definitely has 

Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[jeetu.gol...@gmail.com]

Hello Cyril,

To report back, I tried out the latest RS master codebase from github
with my plugin as per your suggestion. Unfortunately the issue is
still the same.

While I said earlier most of the times (say 90%) the failure occurs
when there are many small bytes of approx 172 bytes following by
something like 7556 bytes or some such number (usually above 3000
bytes).

However, there are instances when the freeze-up occurs with a
relatively short number of bytesmaybe as low as 60 or 70 bytes. So
when I tried running gedit, during startup itself, the ssh client end
froze. While most of the time I will get the main interface window to
the X client and it freezes somewhere along the line, there are also
moments such as these when it just hands early on itself.

I also tried to log in to just a SSH console session via the RS tunnel
plugin and did a "du" on my root directory just to generate a lot of
text traffic in the form of file names scrolling as du goes through my
directory. This worked fine and didn't baulk. However this situation
is a little different from a SSH X forward since here there is only
unidrectional traffic (from ssh server to ssh client). Whereas with a
SSH forward there is a lot of bi-directional traffic. Dunno if that
helps, just trying to get as many clues as possible,

I am also not using any message queue or buffer within my own code, so
as information comes in from ssh client or server I push it down the
pipe via sendItem. Not sure if somehow the information coming in from
these ends is going out of sync and causing a freeze-up.

Your suggestion of this being related to the bug you had closed
wherein the SSL stream would lock up on occasion waiting for another
packed, definitely seemed to fit the profile of the above behaviour.

Is there some other way we can capture additional data from within RS
that would help you confirm or rule out if this problem still persists
or that would help us concretely ascertain if my code is messing up
things or not?

Since the SSH traffic is all encrypted, I am unable to verify if the
data itself gets corrupted somewhere along the line. Though if that
were to happen the SSH client / server should cut off connection with
a bad MAC / bad HMAC / Bad Packet Length or some such message if I
understand it correctly. Though to rule this out I am considering
building a python client and server that will try to push through
controlled data to and fro this pipe, and try to replicate this
freezehopefully something there will then be able to clarify if
data corruption is occurring and is the problem.

As you can say I am fast approaching a state where I am desperate for
ideas ;).any help or suggestion would help my sanity to a great
deal :)

Thank you so much for your time and help :)

Bye for now.



On Mon, Jul 25, 2016 at 11:46 PM, jeetu.gol...@gmail.com
 wrote:
> Hi Cyril,
>
> Thank you so much for taking the time to reply :)
>
>>This does not seem like a crazy amount of traffic either. I'm not sure which 
>>>version of RS you're
>
> I've been working with RS v0.6.0
>
>>using, but a few weeks ago, I fixed a bug that would cause SSL packets to 
>>>hang in the pipeline until
>>a new packet arrives. This could be the reason as your X server is maybe 
>>>waiting for the stream to
>>reach a given point before continuing.
>
> Oh wow :) so there is hope :).I shall download the latest master
> of RS and try again with thatwill keep you posted on the progress
> I make :)
>
>>As I understand, you developed a RS service that streams SSL connection 
>>>binary items through RS
>>friend connections?
>>You're most probably using RsItem objects then. Their size is limited to 
>>>262144 bytes. Any packet
>>larger gets dropped.
>
> Yes I have been using RsItem objects...and this explains so much
> :).is there some other class object I should use wherein this
> limit is not there / or more relaxed? Though at this moment this isn't
> a problem for me since I chunk the data and then send it. However it
> would be good to know what my options here are :)
>
>
> This definitely has given me quite a bit of clarity and hope.I
> shall try with the latest code and well fingers crossed :)will
> report back asap.
>
> Thank you.
>
> Bye for now
>
>
> On Mon, Jul 25, 2016 at 7:51 PM, Cyril Soler
>  wrote:
>> Hi!
>>
>> Congrats for your project!
>> It's nice to see that you managed to develop a plugin! See below some 
>> possible answers to your problem:
>>
>> On 25/07/2016 08:09, jeetu.gol...@gmail.com wrote:
>>> Hi everyone,
>>>
>>> First, let me thank all of you for giving us Retroshare. Works
>>> beautifully :) Thanks.
>>>
>>> I am one of the developers of the open source project eBrainPool
>>> (http://ebrain.in) and for this project I have been toying with a RS
>>> plugin that will allow me to tunnel SSH traffic between RS users. The
>>> idea is to SSH forward X clients tunneled via RS.
>>>
>>> I