Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[Cyril Soler]

you might also want to send additional data through your ssh tunnel, just to 
keep it running. It looks like if you don't SSH needs some kind of
"flush", which we have never cared to implement since RS links are always busy.

I assume you're running a very recent version of the software? Otherwise, you 
need to know that we recently add such a flush on the client side, which
was responsible for RTTs not being fast because they just hanged up waiting for 
more data to come. But that modification is now a few weeks old.

On 10/24/16 13:47, jeetu.gol...@gmail.com wrote:
> Hi guys,
> 
>> ?Anyway have you received my mail which was sent directly to you(instead to 
>> the mailing list?) . Afaik also Gio wrote to you.
> 
> Cave, am sorry I didn't get the mail you sent to me. I did get the
> mail Gio sent, and I replied back to it. It's so nice to get in touch
> with people I met at CCC :)  Could you please re-send me the mail you
> had sent earlier?
> 
> 
>> There's actually a trick to get rid of this problem:
>>
>> - go to PluginsPage.cpp, and comment out line 158: 
>> "ui.enableAll->setEnabled(false);"
>> - restart RS. In the GUI you can now allow to *not check* for plugin hash. 
>> Check it and Quit.
>> - now RS-nogui should load your plugin without checking the hash.
> 
> Wow, this will be super :).will try it over the next few days or
> so and report back.
> 
> My current theory to the issue I am facing is that in the entire
> process where an X client is being forwarded through my RS tunnel, at
> some juncture there is a delay / pause / something, that causes ssh to
> momentarily freeze the desktop while it waits for some more data,
> there are references online I have come across when ssh has been
> reported to cause such momentary or permanent lock ups. Since this
> freeze also locks up RS along with the desktop, this means that the
> next packet of data that would have ordinarily caused things to resume
> doesn't make it through.
> 
> I am not certain of the above theory but it explains why when ssh + X
> server are on the same system as RS there is a freeze, however if ssh,
> RS and X server are not on the same system - either the X server is in
> a VM or in Xephyr, etc, things work well.
> 
> Well hoping retroshare-nogui with my plugin will shed more light.
> 
> Please let me know if any of you think I should try anything else.
> 
> Thank you so much again. Truly appreciate it.
> 
> Bye for now
> 
> 
> 
> 
> On Mon, Oct 24, 2016 at 4:44 PM, Cyril Soler
>  wrote:
>> There's actually a trick to get rid of this problem:
>>
>> - go to PluginsPage.cpp, and comment out line 158: 
>> "ui.enableAll->setEnabled(false);"
>> - restart RS. In the GUI you can now allow to *not check* for plugin hash. 
>> Check it and Quit.
>> - now RS-nogui should load your plugin without checking the hash.
>>
>> On 10/24/16 12:51, Cave wrote:
>>> Dear Jeetu,
>>>
>>> it looks like the hash check for plugins.
>>> I guess you have to allow/approve the new hash once in the gui.
>>>
>>> And then it should also pass the nogui-client check(which seems to miss the 
>>> functionality to ask for approval).
>>>
>>> ?Anyway have you received my mail which was sent directly to you(instead to 
>>> the mailing list?) . Afaik also Gio wrote to you.
>>>
>>>
>>> Best regards
>>> Cave
>>>
>>>
>>> Am 24. Oktober 2016 12:30:36 MESZ, schrieb "jeetu.gol...@gmail.com" 
>>> :
>>>
>>> Hi,
>>>
>>> As stated in my last mail, when I run retroshare-nogui, it rejects my
>>> plugin with the following message :
>>>
>>> RsPluginManager::loadList():
>>>Reference executable hash: 2bf0bcb5ace17e7dfad6681b051ff083cee1e531
>>>Accepted hash: 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea
>>> (WW) Executable hashes do not match. Executable hash has changed.
>>> Discarding the list of accepted/rejected plugins.
>>> Found plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so 
>>> 
>>>   Loading plugin...
>>>   Loading plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so 
>>> 
>>> -> hashing.
>>> -> hash = 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea
>>> -> hash rejected. Giving up plugin.
>>> Examined a total of 1 plugins.
>>> PluginManager: saving list.
>>>   Saving current executable hash: 
>>> 537e4a87038d4b18df08f9ab00b0ef7069e82985
>>>
>>> 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea : REJECTED
>>>
>>>
>>> Is there something I need to do in my plugin to get it to work within
>>> retroshare-nogui?
>>>
>>> I need to test out this plugin with retroshare running in headless mode.
>>>
>>> Would appreciate if someone could help me here.
>>>
>>> Thank you.
>>>
>>> Regards,
>>> Jeetu
>>>
>>>
>>>
>>> On Mon, Oct 17, 2016 at 7:20 PM, jeetu.gol...@gmail.com
>>>  wrote:
>>>
>>> Hi Cyril,
>>>
>>> *Warning 

Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[jeetu.gol...@gmail.com]

Hi guys,

>?Anyway have you received my mail which was sent directly to you(instead to 
>the mailing list?) . Afaik also Gio wrote to you.

Cave, am sorry I didn't get the mail you sent to me. I did get the
mail Gio sent, and I replied back to it. It's so nice to get in touch
with people I met at CCC :)  Could you please re-send me the mail you
had sent earlier?


>There's actually a trick to get rid of this problem:
>
>- go to PluginsPage.cpp, and comment out line 158: 
>"ui.enableAll->setEnabled(false);"
>- restart RS. In the GUI you can now allow to *not check* for plugin hash. 
>Check it and Quit.
>- now RS-nogui should load your plugin without checking the hash.

Wow, this will be super :).will try it over the next few days or
so and report back.

My current theory to the issue I am facing is that in the entire
process where an X client is being forwarded through my RS tunnel, at
some juncture there is a delay / pause / something, that causes ssh to
momentarily freeze the desktop while it waits for some more data,
there are references online I have come across when ssh has been
reported to cause such momentary or permanent lock ups. Since this
freeze also locks up RS along with the desktop, this means that the
next packet of data that would have ordinarily caused things to resume
doesn't make it through.

I am not certain of the above theory but it explains why when ssh + X
server are on the same system as RS there is a freeze, however if ssh,
RS and X server are not on the same system - either the X server is in
a VM or in Xephyr, etc, things work well.

Well hoping retroshare-nogui with my plugin will shed more light.

Please let me know if any of you think I should try anything else.

Thank you so much again. Truly appreciate it.

Bye for now




On Mon, Oct 24, 2016 at 4:44 PM, Cyril Soler
 wrote:
> There's actually a trick to get rid of this problem:
>
> - go to PluginsPage.cpp, and comment out line 158: 
> "ui.enableAll->setEnabled(false);"
> - restart RS. In the GUI you can now allow to *not check* for plugin hash. 
> Check it and Quit.
> - now RS-nogui should load your plugin without checking the hash.
>
> On 10/24/16 12:51, Cave wrote:
>> Dear Jeetu,
>>
>> it looks like the hash check for plugins.
>> I guess you have to allow/approve the new hash once in the gui.
>>
>> And then it should also pass the nogui-client check(which seems to miss the 
>> functionality to ask for approval).
>>
>> ?Anyway have you received my mail which was sent directly to you(instead to 
>> the mailing list?) . Afaik also Gio wrote to you.
>>
>>
>> Best regards
>> Cave
>>
>>
>> Am 24. Oktober 2016 12:30:36 MESZ, schrieb "jeetu.gol...@gmail.com" 
>> :
>>
>> Hi,
>>
>> As stated in my last mail, when I run retroshare-nogui, it rejects my
>> plugin with the following message :
>>
>> RsPluginManager::loadList():
>>Reference executable hash: 2bf0bcb5ace17e7dfad6681b051ff083cee1e531
>>Accepted hash: 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea
>> (WW) Executable hashes do not match. Executable hash has changed.
>> Discarding the list of accepted/rejected plugins.
>> Found plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so 
>> 
>>   Loading plugin...
>>   Loading plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so 
>> 
>> -> hashing.
>> -> hash = 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea
>> -> hash rejected. Giving up plugin.
>> Examined a total of 1 plugins.
>> PluginManager: saving list.
>>   Saving current executable hash: 
>> 537e4a87038d4b18df08f9ab00b0ef7069e82985
>>
>> 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea : REJECTED
>>
>>
>> Is there something I need to do in my plugin to get it to work within
>> retroshare-nogui?
>>
>> I need to test out this plugin with retroshare running in headless mode.
>>
>> Would appreciate if someone could help me here.
>>
>> Thank you.
>>
>> Regards,
>> Jeetu
>>
>>
>>
>> On Mon, Oct 17, 2016 at 7:20 PM, jeetu.gol...@gmail.com
>>  wrote:
>>
>> Hi Cyril,
>>
>> *Warning - a really really long read below, am truly sorry for that*
>>
>> I suggest that you use your SSH tunnel manually: send clear 
>> packets with some ID in them (packet
>> number, whatever you prefer)
>> and make sure that you get them all on the other end of the 
>> tunnel, without major delay. Normally
>> this should work,
>>
>>
>> I have been trying to get my head around this issue (yet). I have 
>> been
>> taking a lot of trials, including what you suggested above. The
>> following are my findings :
>>
>> - I wrote a python client and server to stress test the tunnel in a
>> few ways and didn't find anything untoward. Of course, one thing I
>>   

Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[jeetu.gol...@gmail.com]

Hi,

As stated in my last mail, when I run retroshare-nogui, it rejects my
plugin with the following message :

RsPluginManager::loadList():
   Reference executable hash: 2bf0bcb5ace17e7dfad6681b051ff083cee1e531
   Accepted hash: 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea
(WW) Executable hashes do not match. Executable hash has changed.
Discarding the list of accepted/rejected plugins.
Found plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so
  Loading plugin...
  Loading plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so
-> hashing.
-> hash = 89dc2a00adae7a2b4e7bce01d287b6457b3d8eea
-> hash rejected. Giving up plugin.
Examined a total of 1 plugins.
PluginManager: saving list.
  Saving current executable hash: 537e4a87038d4b18df08f9ab00b0ef7069e82985
  89dc2a00adae7a2b4e7bce01d287b6457b3d8eea : REJECTED


Is there something I need to do in my plugin to get it to work within
retroshare-nogui?

I need to test out this plugin with retroshare running in headless mode.

Would appreciate if someone could help me here.

Thank you.

Regards,
Jeetu



On Mon, Oct 17, 2016 at 7:20 PM, jeetu.gol...@gmail.com
 wrote:
> Hi Cyril,
>
> *Warning - a really really long read below, am truly sorry for that*
>
>>I suggest that you use your SSH tunnel manually: send clear packets with some 
>>ID in them (packet
>>number, whatever you prefer)
>>and make sure that you get them all on the other end of the tunnel, without 
>>major delay. Normally
>>this should work,
>
> I have been trying to get my head around this issue (yet). I have been
> taking a lot of trials, including what you suggested above. The
> following are my findings :
>
> - I wrote a python client and server to stress test the tunnel in a
> few ways and didn't find anything untoward. Of course, one thing I
> would probably like to do at some point is repeat this along with a
> hash of each packet being sent to verify integrity of data. Right now,
> my tests were a little crude.
>
> - SSH X Forwarding through my tunnel works absolutely well if I am
> using the Chromium browser remotely, for some strange reason. It
> routinely fails with gnome-calculator, gedit, libreoffice.
>
>
> - I have been in discussion with the Xorg team too to see if this had
> any co-relation with a possible graphics and / or the X server and
> they have been very helpful and we have tried out various tests to
> check if this was related to GL, etc. Have tried different desktop
> environments - Xfce, Enlightenment,etc. So far all of that has turned
> up empty.
>
>
> - I have collected Wireshark data to see what is happening on the
> wire. These packets show that failure always occurs (so far) when :
>
> SSH Server sends a packet (Usually 118 bytes) -> this gets received by
> the SSH client through the tunnel.
> The SSH Client then sends a packet (Usually 166 bytes) -> this gets
> received by the SSH server through the tunnel
>
> From here on there is a lot of traffic going on where the SSH client
> keeps sending stuff to the RS tunnel port on the same machine, however
> this data doesn't get received by the server.
>
> The SSH server also sends a few packets down the pipe, but these don't
> get received by the client.
>
> *The above is (almost) consistently what I see when the desktop has
> frozen and I stop recording Wireshark data.*
>
> Please feel free to ask for the wireshark data if need be.
>
>
>
> - What is most interesting, and this is what I realized just this
> weekend, is that the following network configurations causes the X
> forwarding over my RS tunnel to predictably and consistently either
> succeed or fail :
>
> Scenario a)
>
> Machine 1 - Desktop with Debian Unstable; acting as X Server; SSH
> client connects to this end of the RS tunnel
>
> Machine 2 - Laptop with Debian Testing; hosts the X clients, SSH
> server is at this end.
>
> Result : If the program being run is Chromium or something simple like
> xterm, sclock all is well, otherwise the X client will surely hang at
> some point.
>
> Scenario b)
>
> Machine 1 (Host VM) - Desktop with Debian Unstable as host;
> Retroshare's GUI and my tunnel plugin will run here; neither SSH
> client nor server will run on this machine.
>
> Machine 2 (Guest VM) - Ubuntu / Debian Testing / Debian Unstable as
> guest in a VM; SSH client will connect from here to the RS tunnel end
> point on Machine 1. X server will be inside the VM.
>
> Machine 3 - Laptop with Debian Testing; hosts the X clients, SSH
> server is at this end.
>
> Result: All applications will work absolutely perfectly without hanging at 
> all.
>
> Scenario c)
>
> Machine 1 (Guest VM) - Ubuntu / Debian Testing / Debian Unstable as
> guest in a VM (OS is same as above); RS GUI and my tunnel plugin will
> run inside the VM; SSH client connects to this end of the RS tunnel;
> X server will be inside this VM.
>
> Machine 2 - Laptop with Debian Testing; hosts the X clients, SSH
> server is at this end.
>
> Result : If the 

Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[jeetu.gol...@gmail.com]

Hi Cyril,

*Warning - a really really long read below, am truly sorry for that*

>I suggest that you use your SSH tunnel manually: send clear packets with some 
>ID in them (packet
>number, whatever you prefer)
>and make sure that you get them all on the other end of the tunnel, without 
>major delay. Normally
>this should work,

I have been trying to get my head around this issue (yet). I have been
taking a lot of trials, including what you suggested above. The
following are my findings :

- I wrote a python client and server to stress test the tunnel in a
few ways and didn't find anything untoward. Of course, one thing I
would probably like to do at some point is repeat this along with a
hash of each packet being sent to verify integrity of data. Right now,
my tests were a little crude.

- SSH X Forwarding through my tunnel works absolutely well if I am
using the Chromium browser remotely, for some strange reason. It
routinely fails with gnome-calculator, gedit, libreoffice.


- I have been in discussion with the Xorg team too to see if this had
any co-relation with a possible graphics and / or the X server and
they have been very helpful and we have tried out various tests to
check if this was related to GL, etc. Have tried different desktop
environments - Xfce, Enlightenment,etc. So far all of that has turned
up empty.


- I have collected Wireshark data to see what is happening on the
wire. These packets show that failure always occurs (so far) when :

SSH Server sends a packet (Usually 118 bytes) -> this gets received by
the SSH client through the tunnel.
The SSH Client then sends a packet (Usually 166 bytes) -> this gets
received by the SSH server through the tunnel

>From here on there is a lot of traffic going on where the SSH client
keeps sending stuff to the RS tunnel port on the same machine, however
this data doesn't get received by the server.

The SSH server also sends a few packets down the pipe, but these don't
get received by the client.

*The above is (almost) consistently what I see when the desktop has
frozen and I stop recording Wireshark data.*

Please feel free to ask for the wireshark data if need be.



- What is most interesting, and this is what I realized just this
weekend, is that the following network configurations causes the X
forwarding over my RS tunnel to predictably and consistently either
succeed or fail :

Scenario a)

Machine 1 - Desktop with Debian Unstable; acting as X Server; SSH
client connects to this end of the RS tunnel

Machine 2 - Laptop with Debian Testing; hosts the X clients, SSH
server is at this end.

Result : If the program being run is Chromium or something simple like
xterm, sclock all is well, otherwise the X client will surely hang at
some point.

Scenario b)

Machine 1 (Host VM) - Desktop with Debian Unstable as host;
Retroshare's GUI and my tunnel plugin will run here; neither SSH
client nor server will run on this machine.

Machine 2 (Guest VM) - Ubuntu / Debian Testing / Debian Unstable as
guest in a VM; SSH client will connect from here to the RS tunnel end
point on Machine 1. X server will be inside the VM.

Machine 3 - Laptop with Debian Testing; hosts the X clients, SSH
server is at this end.

Result: All applications will work absolutely perfectly without hanging at all.

Scenario c)

Machine 1 (Guest VM) - Ubuntu / Debian Testing / Debian Unstable as
guest in a VM (OS is same as above); RS GUI and my tunnel plugin will
run inside the VM; SSH client connects to this end of the RS tunnel;
X server will be inside this VM.

Machine 2 - Laptop with Debian Testing; hosts the X clients, SSH
server is at this end.

Result : If the program being run is Chromium or something simple like
xterm, sclock all is well, otherwise the X client will surely hang at
some point.


Scenario b) had lead me to believe this could be an issue related to X
server, GL / GLX or the graphics drivers running on bare metal with
accelerated graphics, however Scenario c) behaving exactly like
Scenario a) puts a spanner in the works to that theory.

I am unable to see the full picture here. If my tunnel is faulty then
why work in Scenario b)? Why do things fail in configuration such as
Scenario c) and a)?

One  co-relation "could" be that as soon as the X server is the same
as the machine with the RS GUI things fail. Yes I know this is a
stretch.

The RS client is a Qt client and could it be that when the X server
for the remote X client is on the same machine there is a lock up
somewhere causing a freeze. I really don't know how / why this could
bebut am desperate.

To rule out the above I wanted to run RS in non-gui mode with my
plugin coming up and then try out the tests.

When running retroshare-nogui however my plugin doesn't come up. I see
the following on the console :

Found plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so
  Loading plugin...
  Loading plugin /home/jeetu/.retroshare/extensions6/libEmptyPlugin.so
-> hashing.
-> hash = 

Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[jeetu.gol...@gmail.com]

Hi Cyril,

Thanks for the reply.

Yes this sounds like a good strategy. I will try to go ahead and build a
python based program that will stress tests this tunnel with packets of
varying sizes in a bi-directional fashion. Hopefully I will be able to
reproduce this error and we can come closer to understanding where the
problem lies.

Thank you for your help thus far and your offer to look at libretroshare :)

Regards,
Jeetu

On 28 Jul 2016 13:45, "Cyril Soler"  wrote:

> I suggest that you use your SSH tunnel manually: send clear packets with
> some ID in them (packet
> number, whatever you prefer)
> and make sure that you get them all on the other end of the tunnel,
> without major delay. Normally
> this should work,
> but if it does not, and if you're able to create a working prototype that
> shows the bug, I can take
> a look at libretroshare to find the problem for you.
>
> Otherwise, because problems occur mainly when X is used, it might be that
> X expects some particular
> params of the SSH connection (which you get probably when doing "ssh -X or
> ssh -Y" but not using
> your own tunnel system). It's probably worth looking into this.
>
> On 28/07/2016 10:04, jeetu.gol...@gmail.com wrote:
> > Hello Cyril,
> >
> > To report back, I tried out the latest RS master codebase from github
> > with my plugin as per your suggestion. Unfortunately the issue is
> > still the same.
> >
> > While I said earlier most of the times (say 90%) the failure occurs
> > when there are many small bytes of approx 172 bytes following by
> > something like 7556 bytes or some such number (usually above 3000
> > bytes).
> >
> > However, there are instances when the freeze-up occurs with a
> > relatively short number of bytesmaybe as low as 60 or 70 bytes. So
> > when I tried running gedit, during startup itself, the ssh client end
> > froze. While most of the time I will get the main interface window to
> > the X client and it freezes somewhere along the line, there are also
> > moments such as these when it just hands early on itself.
> >
> > I also tried to log in to just a SSH console session via the RS tunnel
> > plugin and did a "du" on my root directory just to generate a lot of
> > text traffic in the form of file names scrolling as du goes through my
> > directory. This worked fine and didn't baulk. However this situation
> > is a little different from a SSH X forward since here there is only
> > unidrectional traffic (from ssh server to ssh client). Whereas with a
> > SSH forward there is a lot of bi-directional traffic. Dunno if that
> > helps, just trying to get as many clues as possible,
> >
> > I am also not using any message queue or buffer within my own code, so
> > as information comes in from ssh client or server I push it down the
> > pipe via sendItem. Not sure if somehow the information coming in from
> > these ends is going out of sync and causing a freeze-up.
> >
> > Your suggestion of this being related to the bug you had closed
> > wherein the SSL stream would lock up on occasion waiting for another
> > packed, definitely seemed to fit the profile of the above behaviour.
> >
> > Is there some other way we can capture additional data from within RS
> > that would help you confirm or rule out if this problem still persists
> > or that would help us concretely ascertain if my code is messing up
> > things or not?
> >
> > Since the SSH traffic is all encrypted, I am unable to verify if the
> > data itself gets corrupted somewhere along the line. Though if that
> > were to happen the SSH client / server should cut off connection with
> > a bad MAC / bad HMAC / Bad Packet Length or some such message if I
> > understand it correctly. Though to rule this out I am considering
> > building a python client and server that will try to push through
> > controlled data to and fro this pipe, and try to replicate this
> > freezehopefully something there will then be able to clarify if
> > data corruption is occurring and is the problem.
> >
> > As you can say I am fast approaching a state where I am desperate for
> > ideas ;).any help or suggestion would help my sanity to a great
> > deal :)
> >
> > Thank you so much for your time and help :)
> >
> > Bye for now.
> >
> >
> >
> > On Mon, Jul 25, 2016 at 11:46 PM, jeetu.gol...@gmail.com
> >  wrote:
> >> Hi Cyril,
> >>
> >> Thank you so much for taking the time to reply :)
> >>
> >>> This does not seem like a crazy amount of traffic either. I'm not sure
> which >version of RS you're
> >> I've been working with RS v0.6.0
> >>
> >>> using, but a few weeks ago, I fixed a bug that would cause SSL packets
> to >hang in the pipeline until
> >>> a new packet arrives. This could be the reason as your X server is
> maybe >waiting for the stream to
> >>> reach a given point before continuing.
> >> Oh wow :) so there is hope :).I shall download the latest master
> >> of RS and try again with thatwill keep you 

Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[Cyril Soler]

I suggest that you use your SSH tunnel manually: send clear packets with some 
ID in them (packet
number, whatever you prefer)
and make sure that you get them all on the other end of the tunnel, without 
major delay. Normally
this should work,
but if it does not, and if you're able to create a working prototype that shows 
the bug, I can take
a look at libretroshare to find the problem for you.

Otherwise, because problems occur mainly when X is used, it might be that X 
expects some particular
params of the SSH connection (which you get probably when doing "ssh -X or ssh 
-Y" but not using
your own tunnel system). It's probably worth looking into this.

On 28/07/2016 10:04, jeetu.gol...@gmail.com wrote:
> Hello Cyril,
>
> To report back, I tried out the latest RS master codebase from github
> with my plugin as per your suggestion. Unfortunately the issue is
> still the same.
>
> While I said earlier most of the times (say 90%) the failure occurs
> when there are many small bytes of approx 172 bytes following by
> something like 7556 bytes or some such number (usually above 3000
> bytes).
>
> However, there are instances when the freeze-up occurs with a
> relatively short number of bytesmaybe as low as 60 or 70 bytes. So
> when I tried running gedit, during startup itself, the ssh client end
> froze. While most of the time I will get the main interface window to
> the X client and it freezes somewhere along the line, there are also
> moments such as these when it just hands early on itself.
>
> I also tried to log in to just a SSH console session via the RS tunnel
> plugin and did a "du" on my root directory just to generate a lot of
> text traffic in the form of file names scrolling as du goes through my
> directory. This worked fine and didn't baulk. However this situation
> is a little different from a SSH X forward since here there is only
> unidrectional traffic (from ssh server to ssh client). Whereas with a
> SSH forward there is a lot of bi-directional traffic. Dunno if that
> helps, just trying to get as many clues as possible,
>
> I am also not using any message queue or buffer within my own code, so
> as information comes in from ssh client or server I push it down the
> pipe via sendItem. Not sure if somehow the information coming in from
> these ends is going out of sync and causing a freeze-up.
>
> Your suggestion of this being related to the bug you had closed
> wherein the SSL stream would lock up on occasion waiting for another
> packed, definitely seemed to fit the profile of the above behaviour.
>
> Is there some other way we can capture additional data from within RS
> that would help you confirm or rule out if this problem still persists
> or that would help us concretely ascertain if my code is messing up
> things or not?
>
> Since the SSH traffic is all encrypted, I am unable to verify if the
> data itself gets corrupted somewhere along the line. Though if that
> were to happen the SSH client / server should cut off connection with
> a bad MAC / bad HMAC / Bad Packet Length or some such message if I
> understand it correctly. Though to rule this out I am considering
> building a python client and server that will try to push through
> controlled data to and fro this pipe, and try to replicate this
> freezehopefully something there will then be able to clarify if
> data corruption is occurring and is the problem.
>
> As you can say I am fast approaching a state where I am desperate for
> ideas ;).any help or suggestion would help my sanity to a great
> deal :)
>
> Thank you so much for your time and help :)
>
> Bye for now.
>
>
>
> On Mon, Jul 25, 2016 at 11:46 PM, jeetu.gol...@gmail.com
>  wrote:
>> Hi Cyril,
>>
>> Thank you so much for taking the time to reply :)
>>
>>> This does not seem like a crazy amount of traffic either. I'm not sure 
>>> which >version of RS you're
>> I've been working with RS v0.6.0
>>
>>> using, but a few weeks ago, I fixed a bug that would cause SSL packets to 
>>> >hang in the pipeline until
>>> a new packet arrives. This could be the reason as your X server is maybe 
>>> >waiting for the stream to
>>> reach a given point before continuing.
>> Oh wow :) so there is hope :).I shall download the latest master
>> of RS and try again with thatwill keep you posted on the progress
>> I make :)
>>
>>> As I understand, you developed a RS service that streams SSL connection 
>>> >binary items through RS
>>> friend connections?
>>> You're most probably using RsItem objects then. Their size is limited to 
>>> >262144 bytes. Any packet
>>> larger gets dropped.
>> Yes I have been using RsItem objects...and this explains so much
>> :).is there some other class object I should use wherein this
>> limit is not there / or more relaxed? Though at this moment this isn't
>> a problem for me since I chunk the data and then send it. However it
>> would be good to know what my options here are :)
>>
>>
>> This definitely has 

Re: [Retroshare-devel] Help needed debugging my plugin that enables RS to act as a SSH tunnel

[jeetu.gol...@gmail.com]

Hello Cyril,

To report back, I tried out the latest RS master codebase from github
with my plugin as per your suggestion. Unfortunately the issue is
still the same.

While I said earlier most of the times (say 90%) the failure occurs
when there are many small bytes of approx 172 bytes following by
something like 7556 bytes or some such number (usually above 3000
bytes).

However, there are instances when the freeze-up occurs with a
relatively short number of bytesmaybe as low as 60 or 70 bytes. So
when I tried running gedit, during startup itself, the ssh client end
froze. While most of the time I will get the main interface window to
the X client and it freezes somewhere along the line, there are also
moments such as these when it just hands early on itself.

I also tried to log in to just a SSH console session via the RS tunnel
plugin and did a "du" on my root directory just to generate a lot of
text traffic in the form of file names scrolling as du goes through my
directory. This worked fine and didn't baulk. However this situation
is a little different from a SSH X forward since here there is only
unidrectional traffic (from ssh server to ssh client). Whereas with a
SSH forward there is a lot of bi-directional traffic. Dunno if that
helps, just trying to get as many clues as possible,

I am also not using any message queue or buffer within my own code, so
as information comes in from ssh client or server I push it down the
pipe via sendItem. Not sure if somehow the information coming in from
these ends is going out of sync and causing a freeze-up.

Your suggestion of this being related to the bug you had closed
wherein the SSL stream would lock up on occasion waiting for another
packed, definitely seemed to fit the profile of the above behaviour.

Is there some other way we can capture additional data from within RS
that would help you confirm or rule out if this problem still persists
or that would help us concretely ascertain if my code is messing up
things or not?

Since the SSH traffic is all encrypted, I am unable to verify if the
data itself gets corrupted somewhere along the line. Though if that
were to happen the SSH client / server should cut off connection with
a bad MAC / bad HMAC / Bad Packet Length or some such message if I
understand it correctly. Though to rule this out I am considering
building a python client and server that will try to push through
controlled data to and fro this pipe, and try to replicate this
freezehopefully something there will then be able to clarify if
data corruption is occurring and is the problem.

As you can say I am fast approaching a state where I am desperate for
ideas ;).any help or suggestion would help my sanity to a great
deal :)

Thank you so much for your time and help :)

Bye for now.



On Mon, Jul 25, 2016 at 11:46 PM, jeetu.gol...@gmail.com
 wrote:
> Hi Cyril,
>
> Thank you so much for taking the time to reply :)
>
>>This does not seem like a crazy amount of traffic either. I'm not sure which 
>>>version of RS you're
>
> I've been working with RS v0.6.0
>
>>using, but a few weeks ago, I fixed a bug that would cause SSL packets to 
>>>hang in the pipeline until
>>a new packet arrives. This could be the reason as your X server is maybe 
>>>waiting for the stream to
>>reach a given point before continuing.
>
> Oh wow :) so there is hope :).I shall download the latest master
> of RS and try again with thatwill keep you posted on the progress
> I make :)
>
>>As I understand, you developed a RS service that streams SSL connection 
>>>binary items through RS
>>friend connections?
>>You're most probably using RsItem objects then. Their size is limited to 
>>>262144 bytes. Any packet
>>larger gets dropped.
>
> Yes I have been using RsItem objects...and this explains so much
> :).is there some other class object I should use wherein this
> limit is not there / or more relaxed? Though at this moment this isn't
> a problem for me since I chunk the data and then send it. However it
> would be good to know what my options here are :)
>
>
> This definitely has given me quite a bit of clarity and hope.I
> shall try with the latest code and well fingers crossed :)will
> report back asap.
>
> Thank you.
>
> Bye for now
>
>
> On Mon, Jul 25, 2016 at 7:51 PM, Cyril Soler
>  wrote:
>> Hi!
>>
>> Congrats for your project!
>> It's nice to see that you managed to develop a plugin! See below some 
>> possible answers to your problem:
>>
>> On 25/07/2016 08:09, jeetu.gol...@gmail.com wrote:
>>> Hi everyone,
>>>
>>> First, let me thank all of you for giving us Retroshare. Works
>>> beautifully :) Thanks.
>>>
>>> I am one of the developers of the open source project eBrainPool
>>> (http://ebrain.in) and for this project I have been toying with a RS
>>> plugin that will allow me to tunnel SSH traffic between RS users. The
>>> idea is to SSH forward X clients tunneled via RS.
>>>
>>> I