On Tue, 1 Dec 2009, Peter Stuge wrote:
I know that's how SFTP works and I still don't understand how that protocol
design prevents our API/ABI to work.
Trace through the code then. Use sftp_write.c but increase buffer size to
>=32768 and you will notice the problem with a lower layer processing only
part of what is a packet in a higher layer.
That's *a bug* and not necessarily a design flaw that we cannot fix within our
current API/ABI. I'm convinced we can fix it.
The fact that send() and recv() return such values, and all SSL libraries
I've used return such numbers for their SSL_send()/SSL_recv() equivalents
tell me that's a well used paradigm that I think we should keep.
Sure, if it works. But you should also be prepared to change it if
that makes sense. SSL is a stream but SSH transport and SFTP aren't.
The read/write functions we use for sftp do their best to look like streams
and I think we should maintain that.
The current way of doing it still works. It's just a bit lame with buffer
sizes.
Define lame? Play with the sftp buffer size and you'll notice the problem.
It doesn't really work.
I mean lame in the meaning that it has lots of room for improvement. The not
working part is a bug that we can and should fix.
We will always have lots of state within libssh2.
Things could be changed around to keep less state in libssh2 without losing
any functionality, e.g. by having applications prepare packets before they
send them.
Yes, but that would require a different API that then probably instead would
hand over more logic on the app which might make the API slightly more
involved to use which isn't necessarily a good thing. Although I'll agree that
we could most probably come up with API changes that will make several things
more elegant, both for apps and for the library implementation.
And if you trusted select() for blocking fds then the required state could
be reduced even further.
Now we're looping into a discussion we've had before, and I still don't buy
that argument.
There's a big difference in that TCP is a stream, so it's fine for the stack
to send fewer bytes than requested. SSH is a packet transport and complete
packets must always be sent.
Very true, and it makes my comparison somewhat bad. But we don't expose the
API as "send a packet", we expose the API to allow the app to send bytes. In
what looks like a stream to the app. And besides, even things like TLS are
packet-based and still everyone exposes TLS sending and receiving as streams.
We abstract what's actually happening by providing an API that makes sense to
apps and allows enough power to do what they want. I don't think we have to be
completely transparent with regarding to all the SSH concepts.
The simplest way out for the library is to give applications access to the
real number of bytes that need to be sent by the lowest layer. That is not
to say it is neccessarily the best way.
We could also have libssh2 just build packages and let the application send()
it, and let the app recv() data and have libssh2 decode the binary lumps.
Any other dealing with 0 is a bug in my eyes.
It only works when all lower layer extra bytes come before upper layer
bytes. This isn't the case, I think I mentioned transport layer padding and
MAC which both are appendeed after the upper layer bytes. There may be more
instances of this. The library then has to lie either about bytes in the
beginning or in the end, I don't think that's very pretty either way.
I disagree again about it not working.
It works already and it still works (or can work) no matter who adds what
when. The trick is to never claim that the payload is sent until all the lower
layer bits are also sent, even if those lower layer bits are sent after the
payload data passed on from the upper layers. It may not be pretty all over,
but it works.
In my eyes you're fighting the current API with a lot of energy to point out
why we should do something (completely) different. I'm not arguing against
that a completely different API might be better in many ways, and I would be
happy to work on and discuss how a new and improved API would work. But at the
same time, we have an API today and with all its flaws and unfeatures it is
there and it is being used and we can make it work the way it is documented.
And I think it is worth bugfixing and make/keep working so that exiting apps
keep working and new apps can be written to use libssh2 all along until the
future day when a brand new and remodeled API can be introduced.
Given the lack of busloads of people involved here, I fear that a new API may
take a while to get done.
--
/ daniel.haxx.se
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
