Hi, 14.04.2009 09:23, Kern Sibbald wrote: > Hello James, > > It is hard to respond because I don't know what a PV driver is or how it is > related to network drivers.
Neither do I :-) > On Tuesday 14 April 2009 08:41:37 James Harper wrote: >> I'm writing Windows PV drivers for Xen, and bacula is the only app that >> I have found that can reliably trigger a certain bug in the network >> driver. While debugging that, I have noticed that bacula may not be >> sending data in the most efficient way possible... > > That depends on how you define "the most efficient way possible". > > Bacula makes a large number of small network I/O calls, but it expects the OS > to deal with getting them efficiently across the network. > That said, there were some important optimizations between 2.2.8 and 3.0.0, > which reduce the number of network writes by a factor of 2. This seems to be present in 2.5.4x as I can't see what James describes in my test setup. >> Basically, when my drivers receive a packet, the packet consists of a >> whole bunch of buffers chained together. The packets I get from Bacula >> contain quite a few small chunks, eg a sample packet consists of a large >> number of 4 byte and 12 byte buffers (15 of each in a single 10k TCP >> packet). Not here... I see some exchange of small packets when a new file is processed, but it's far from 15 packets. I just used WireShark to check, and found this (all FD->SD, I didn't look for the other direction): 143 octets (File name and stats) 10 octets (whatever) 1260 octets (repeated, file data, 1314 octets on the wire, 100MBit, no Jumbo frames) In other words, the network traffic seems to be quite efficient since mostly large packets are sent. For some reason the MTU is not fully used (set to 1514 octets on Windows end, 1500 on linux end), but I don't care much for that now. If anybody has an idea, though... > It's fairly easy to see the performance difference of using >> smaller buffer sizes by using the iperf tool. I'm not sure if Linux >> handles things in the same way though so this might not apply there... > > I am not quite sure what you mean by small buffer sizes. There are several > different parameters that one must clearly distinguish. Bacula does send > network I/O in sizes that are convenient for it. These are not "buffered", > but simply reads and writes of the socket. This involves a certain number of > small I/O sizes (4 bytes +). However, once a file is being transferred, > Bacula packs the file into network buffers and sends as much as it can. The > network buffer size is determined by what the OS permits. By default Bacula > starts with a 64K size and reduces it if the OS indicates that it is too > large. This size can be defined by the user to optimize I/O performance. > > Perhaps across the Internet the small read/write requests could be > inefficient, but within a 100Mb or Gigabit network, they don't have much > importance (as far as I know). Well, I think the relative overhad would hurt more for slower connections, but saturating a GBit-link with jumbo frames with user data would be nice, too. But my observations don't indicate there is a real bottleneck here. > >> I think that there could be some performance benefit in keeping a >> per-socket buffer and only sending data onto the wire when we have >> enough, at least for the fd->sd connection. Has this been considered >> before? > > There are very few circumstances in Bacula when you can buffer socket data, > because most of the communications are interactive -- that is Bacula sends > some data and expects a response. During saving or restoring a file, Bacula > packs the data into buffers (the size can be determined by the user). > > We would be happy to consider something more, but I would recommend two > things: > > 1. Do your testing with version 3.0.0. Seems well worth it, considering the difference of my observations to what James reports. > 2. Test real file transfers and get some performance data that shows that it > is really a bottle neck. At least for a local network, I am sure you will > find that the comm part of Bacula is not the restraining factor. Agreed :-) Arno >> The system I'm seeing this on is running 2.2.8 so it's pretty >> old... maybe buffering has already been implemented in later versions? > > Buffering of file data is implemented in all versions (the user can control > the sizes, so some experimenting is needed -- perhaps on your network our > defaults are not optimum). Reducing the number of socket writes by a factor > of two was implemented in 3.0.0 (possibly before). > >> (A quick look at bnet.c and bsock.c doesn't reveal anything obvious >> though...) >> > > Unfortunately, not much is evident in Bacula -- simply because what it is > doing is rather complicated. > > Regards, > > Kern > -- Arno Lehmann IT-Service Lehmann Sandstr. 6, 49080 Osnabrück www.its-lehmann.de ------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel
