I think that’s why go should be driven by the community but not by Bell labs nor Google labs. Works in Bell Labs or Google Labs may leave the job, and may not know everything. But the community have enough Human Resources and various of knowledges.
Some scientists and reaserchers worked for commercial company for whole of their life, but never do a things owned by themselves. Like Anders Hejlsberg, the architect of Delphi and C#. What a pity. I think go programming language should also belongs to the community but not Google. One day Bell Lab or Google are closed, the community still exists. > On Mar 11, 2020, at 20:43, Nicola Murino <nicola.mur...@gmail.com> wrote: > > Hi all, > > I want to share the performances analysis I recently did for SFTPGo, the > fully featured and highly configurable SFTP server written in Go > (https://github.com/drakkan/sftpgo). > > When I decided to write an SFTP server I evaluated the available libraries > and I did some quick performance tests too. > > Golang's SSH stack and pkg/sftp were able to easily saturate a Gigabit > connection and this seemed enough. I noticed that OpenSSH was faster, but I > didn't investigate further. > > So I chose Golang for SFTPGo. > > The project is growing fast and one of the users also noticed that SFTPGo has > lower performance than OpenSSH. He opened an issue providing some stats when > using a 40Gb Ethernet card. > > I did some more profiling and discovered that the main bottlenecks are, > unsurprisingly, the cipher used and the message authentication. So we can > have a huge performance boost using a fast cipher with implicit messages > authentication, for example aes128-...@openssh.com, however this cipher is > not widely supported. > > The most used cipher, and the one used in the user's tests is AES-CTR, and > the Golang implementations seems quite slow. > > He noticed that an unmerged patch is available for Golang, greatly improving > AES-CTR performance: > > https://go-review.googlesource.com/c/go/+/51670 > > I applied this patch and, while performance improved, the AES-CTR SFTP > transfers were still slower than the AES-GCM ones. The bottleneck is now the > MAC computation. > > The tested hardware supports Intel SHA extensions but Golang's SHA256 > implementation only uses the AVX2 extension. > > Again, I was lucky: I can simply use minio/sha256-simd as a drop-in > replacement for Golang's SHA256 implementation: > > https://github.com/drakkan/crypto/commit/17ef3d252b1c9d6124fa17889033e11eaa5c9ddb > > The performance improved again, but OpenSSH was still faster. > > To my great surprise I noticed that my very simple SCP implementation > (https://github.com/drakkan/sftpgo/blob/master/sftpd/scp.go) was now as fast > as OpenSSH! > > So this time I have to look at pkg/sftp: I found some extraneous > copies/allocations in critical paths and I sent some pull requests that are > now merged in the master branch. Still, SFTP transfers were slower than > OpenSSH ones. > > Compared to my SCP implementation the main difference is that pkg/sftp > allocates a new slice for each SFTP packet, while my SCP implementation > allocates a slice once and then reuses this slice. > > Basically for each SFTP packet pkg/sftp does something like this: > > data := make([]byte, size) > > So I wrote a proof of concept allocator that tries to avoid all these extra > allocations reusing the previously allocated slices: > > https://github.com/drakkan/sftp/blob/master/allocator.go > > And bingo! Now SFTPGo performance is very close to OpenSSH! You can find the > full benchmark results here: > > https://github.com/drakkan/sftpgo/blob/master/docs/performance.md > > Conclusion: I see several complaints about Go performance, especially > compared to Rust, but, at least in my use case, Go can be as fast as a C > project (such as OpenSSH). But some special attention is required and thus > this improved performance is not by default available to all the users. > > Now some questions: > > 1) for the pkg/sftp allocator I'm discussing with pkg/sftp maintainers to > find a way to get it included. Do you have smarter ideas to avoid these > allocations? > 2) There is a patch available for AES-CTR in Golang (since 2017): I wonder > why it is not yet merged? > 3) To improve SHA computation performance, I cannot find anything usable in > Golang itself. Is there any plan to have support for Intel SHA Extensions and > AVX512 directly in Golang anytime soon? > > Thank you for this great programming language, it makes it really simple to > add new features to SFTPGo! > > regards, > Nicola > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com > <mailto:golang-nuts+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/7cfdd60d-7415-4c2f-bd7d-9d0372461a3c%40googlegroups.com > > <https://groups.google.com/d/msgid/golang-nuts/7cfdd60d-7415-4c2f-bd7d-9d0372461a3c%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/A06728A5-EBC1-4B2A-B5A0-D2DE526ACE30%40icloud.com.
