[linrad] Re: Linrad Network

2007-08-28 Thread Leif Asbrink
Hello Joe,

 I've been experimenting with the Linrad Network.  All seems well using 
 options 4, 5, 6, or 7, but when I turn on option 8 (Send timf2) or option 9 
 (send fft2 transforms) the AFC graph and S meter graphs become very slow and 
 jerky and the main spectrum and high resolution graphs seem to slow down 
 some.  If I try to X out, the screen freezes and I have to close the 
 window and restart Linrad.  CPU utilization is low, so I'm wondering if 
 something is causing a conflict in the network processing thread for options 
 8 and 9?  I'm using WinXP on a P4 with 1.5G RAM and Linrad 2.35.
OK. I tested for this and I can reproduce the error.


 Consistent 
 results on my laptop as well.  I haven't checked how it behaves under Linux 
 yet, but I have a hunch it's the same.
No, I can not see this problem under Linux.

The network is complicated because each time a big FFT has been completed
a lot of data is released in a big chunk. Linrad has to send the data in 
small packets that are evenly distributed in time because the network 
does not like to be heavily loaded some times and idle in between. 
I have forgotten the exact details, but packet losses was the problem.
Maybe it was the switches in my network that required this complication.

I will look for the error. In the meantime you might run a first 
instance of Linrad that sends raw data to the network as well as
timf2. You can then use a second instance of Linrad in which
you hopefully will be able to click on signals and see normal
behaviour of S-meter and AFC. 

73

Leif / SM5BSZ

#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Re: MAP65 and Linrad Network

2007-06-22 Thread Leif Asbrink
Hi Joe,

 Yes, it should be possible to run everything on a single 
 fast computer with plenty of memory.  I have not yet 
 established that the necessary CPU-sharing during 
 time-critical parts of each program is handled adequately 
 for this to work without glitches.  Neither Linrad nor 
 Windows is a very good real-time O/S, and one must work 
 around their limitations in this area.
I do not think there would be any problems:-)

 Moreover, one will almost certainly want separate screens 
 for Linrad and MAP65 -- both of which generally use most or 
 all of a normal-size screen.
Yes, I agree - but those who have only one standard computer
should know it is possible - if it really is;-)


 For testing MAP65 when real signals are not available, and 
 anyway so that I can get 100% repeatable results, I solved 
 this problem in a slightly different way.  I saved some data 
 by using the Linrad S command during the ARRL EME contest 
 last November.  With a slightly modified Linrad I converted 
 the data to 16-bit TIMF2 format, and wrote it to a file.  A 
 simple program that I call plrs (for pseudo-Linrad send) 
 can read this file and multicast it in the same way that 
 Linrad would do, so that MAP65 can receive it.  The data 
 file, 11 minutes of original data, amounts to 507 MB.  After 
 compression with bzip2 it is 228 MB.  I will make the data 
 file available soon, together with plrs and MAP65, to anyone 
 who wants to participate in testing MAP65.
Hmmm, I would have liked a file that allows me to play with
Linrad on the data that are sent to MAP65. Presumably you have
a couple of spurs and perhaps other interesting interference
that MAP65 (or at least the waterfall graph) would benefit from
not getting. 

Surely, for MAP65 testing and development I agree that your
strategy is more convenient - but from my point of view the
original file is far more interesting

73  Leif / SM5BSZ

#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Re: MAP65 and Linrad Network

2007-06-22 Thread Joe Taylor

Hi Leif,

Moreover, one will almost certainly want separate screens 
for Linrad and MAP65 -- both of which generally use most or 
all of a normal-size screen.


Yes, I agree - but those who have only one standard computer
should know it is possible - if it really is;-)


When MAP65 is available, I will be sure to document how it 
can best be used, and what the hardware requirements will be.



Hmmm, I would have liked a file that allows me to play with
Linrad on the data that are sent to MAP65. Presumably you have
a couple of spurs and perhaps other interesting interference
that MAP65 (or at least the waterfall graph) would benefit from
not getting. 


This will not be a problem.  I have the original raw data 
file and have placed it at 
http://physics.princeton.edu/pulsar/K1JT/06_0744.raw.bz2


It is 263 MB in size.

One of the nuisances of using the original data file is that 
Linrad's raw data are not time-stamped.  As you know, JT65 
transmissions must start at the top of a UTC minute.  To use 
the raw data file effectively with MAP65 you will need to do 
something to ensure that the multicast packets contain times 
that are close to being correct with respect to the 
original time (modulo 60 seconds).



Surely, for MAP65 testing and development I agree that your
strategy is more convenient - but from my point of view the
original file is far more interesting


Yes.  It probably will be far more interesting, for you 
especially.  You will find plenty of birdies and other junk 
in the test file!!  See 
http://physics.princeton.edu/pulsar/K1JT/MAP65_1.JPG .


-- Joe, K1JT

#
This message is sent to you because you are subscribed to
 the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Re: MAP65 and Linrad Network

2007-06-22 Thread Leif Asbrink
Hi Joe,

 I have the original raw data file and have placed it at 
 http://physics.princeton.edu/pulsar/K1JT/06_0744.raw.bz2
Excellent:-)

 One of the nuisances of using the original data file is that 
 Linrad's raw data are not time-stamped.  
Oooh! But they are:-) Your file is stamped 1163231107.413487 which
was the output of the routine current_time() at the start of
the recording.
 
double current_time(void)
{
struct timeval t;
gettimeofday(t,NULL);
return 0.01*t.tv_usec+t.tv_sec;
}

This is the time in seconds with six decimals since Epoch
and if you run Linrad on the file with a fast waterfall
you will see the time on the waterfall starting at 07.45.08

Unfortunately I have missed to put the correct time in the 
time variable of the network packets. Currently the time
is the time when data was read from the hard disk and not
when it was stored there. I will correct for the next version:-)

The 'S' file only has a single time stamp for when write was
started. With Delta 44 soundcards, the sampling rate is 
pretty accurately 96 kHz (96014 Hz with the card in this 
particular computer.) I think one can safely assume that
the error is well below 0.05% so in a 1 hour recording
the time error at the end should stay well below 2 seconds.

 As you know, JT65 
 transmissions must start at the top of a UTC minute.  To use 
 the raw data file effectively with MAP65 you will need to do 
 something to ensure that the multicast packets contain times 
 that are close to being correct with respect to the 
 original time (modulo 60 seconds).
OK. I hope that you have found the time to be correct when
receiving timf2 data from Linrad in real time.

  Surely, for MAP65 testing and development I agree that your
  strategy is more convenient - but from my point of view the
  original file is far more interesting
 
 Yes.  It probably will be far more interesting, for you 
 especially.  You will find plenty of birdies and other junk 
 in the test file!!  See 
 http://physics.princeton.edu/pulsar/K1JT/MAP65_1.JPG .
Fine!

I do not know how to interpret the MAP65 screen. The polarisation
data of AA1YN seems odd to me. They change very rapidly:
0746:  90
0748: 135
0750: 135
0752: 135
0754:   0
0756:  45

Which signal on the screen that has this behaviour is not obvious to me.
The processing screen says Freq 128 and DF=0 (or very close.)
When I look at the file, I find the weak signal at the center of the 
lower waterfall to start at 144.129823 and drift down to 144.129814.
That is a 1.818 Hz frequency shift as compared to the MAP65 screen???
There is some info Options in the upper waterfall, but that does not 
fit. The signal at 144.12982 is right hand circular in Linrad and it 
does not change polarisation during the 10 minutes.

The strong signal at -880 Hz in the lower waterfall is at 144.12894
in Linrad. This signal is elliptic, about 50% left hand. Polarisation
does not change with time on this signal either.

How do I compute the true frequency of the sync tone from the data 
in the MAP65 screen? 

73

Leif / SM5BSZ


#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Re: MAP65 and Linrad Network

2007-06-22 Thread Joe Taylor

Hi Leif,

The combination works OK, and but a significant number of 
packets (around 400 in each minute, or slightly more than 1% 
of the data) are dropped during MAP65's most 
compute-intensive parts of each minute.  Since plrs does no 
significant computing, I imagine that the problem may be 
worse when running Linrad + MAP65.


It could also be the other way around.


Indeed, it could.  In that case I will be surprised, but it will 
hardly be the first time I have been similarly surprised.



It may be necessary to add a call to Sleep(0) under Windows
or to sched_yield() under Linux at regular intervals in all
routines that might lock up the CPU for a too long time.

Those calls are effected by lir_sched_yield() in the OS independent
code of Linrad and I found it necessary to place about 45 such
calls within Linrad.


Hmmm, I may have trouble doing the equivalent.  For example, 
MAP65 does some very long FFTs using tthe FFFTW3 library.  I 
can't very well put calls to relinquish CPU control inside these 
tasks.  On the other hand, there is a threads version of 
FFTW3, which might be what's needed.


Right now such issues are not a top priority, though.

-- Joe, K1JT

#
This message is sent to you because you are subscribed to
 the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] MAP65 and Linrad Network

2007-06-21 Thread Joe Taylor

MAP65 Status


I have now established the necessary network connection 
between Linrad and MAP65.  So far, I have found nothing to 
prevent a happy marriage!


MAP65 receives a full 90 kHz of xpol signal from Linrad, 
finds all the JT65 signals in that bandwidth, matches the 
linear polarization angle of each one, decodes the messages, 
and provides the operator with a band map showing 
callsigns, operating frequencies, polarization angles, and 
message contents over the past 15 minutes or so.  The 
program provides a full-width waterfall display and another 
one zoomed in on the frequency of the station currently 
being worked.



Questions about Linrad's TIMF2 Multicast Data
-

Header information accompanying Linrad's multicast TIMF2 
data includes two single-byte parameters, userx_no and 
passband_direction, about which I have questions.  I 
understand that userx_no should correspond to the number of 
RF channels, with negative sign indicating floating format. 
 I would have expected the value -4 for my system using 
xpol antennas and the WSE converters (I and Q for each of 
two polarizations), but in fact I see -2.  Is this as it 
should be?


Similarly, I would have expected passband_direction=1 as I 
have no LO's on the high side and the spectrum is not 
inverted.  However, Linrad sends passband_direction=-1.  Is 
this correct?



Linrad FFT Versions
---

I have been using FFT Version 0 for the first forward, first 
backward, and second forward FFTs.  This produces 
floating-point TIMF2 data on the multicast network.


To save on memory usage in MAP65 it may be desirable to use 
16-bit data for TIMF2.  I have effected this by setting 
first forward FFT to version 5, first backward FFT to 
version 1, and second forward FFT to version 2.  Are these 
good choices?  Is there any downside to their use that I 
might not have thought about?  Also, with these settings I 
notice that the signal in the high resolution graph (red dB 
lines) is higher than it was with FFT versions 0.  Should I 
adjust some other parameter to bring this level down be 
several dB?



MAP65 with Windows, Linux, FreeBSD, Winrad, SDRxxx ... ?


Like WSJT, MAP65 runs on both Windows and Linux.  (It should 
also run on FreeBSD and Macintosh, but I can't test those 
here.)  In my station I will probably run Linrad under Linux 
 and MAP65 on Windows, on a second computer.


In response to questions I have been getting: MAP65 could 
run with Winrad or with various SDR's if their software is 
modified to provide the necessary multicasting capability. 
In itself, this would not be too difficult.


However, one of the most important MAP65 features is its 
automatic Rx-polarization-matching capability.  That 
requires a receiver with full xpol processing, which (as far 
as I know) is not now provided by Winrad, SDR-1000, SDR-14, 
or SDR-IQ.


-- 73, Joe, K1JT

#
This message is sent to you because you are subscribed to
 the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Re: MAP65 and Linrad Network

2007-06-21 Thread Leif Asbrink
Hello Joe,

 I have now established the necessary network connection 
 between Linrad and MAP65.  So far, I have found nothing to 
 prevent a happy marriage!

 MAP65 receives a full 90 kHz of xpol signal from Linrad, 
 finds all the JT65 signals in that bandwidth, matches the 
 linear polarization angle of each one, decodes the messages, 
 and provides the operator with a band map showing 
 callsigns, operating frequencies, polarization angles, and 
 message contents over the past 15 minutes or so.  The 
 program provides a full-width waterfall display and another 
 one zoomed in on the frequency of the station currently 
 being worked.

Excellent!!


 
 
 Questions about Linrad's TIMF2 Multicast Data
 -
 
 Header information accompanying Linrad's multicast TIMF2 
 data includes two single-byte parameters, userx_no and 
 passband_direction, about which I have questions.  I 
 understand that userx_no should correspond to the number of 
 RF channels, with negative sign indicating floating format. 
   I would have expected the value -4 for my system using 
 xpol antennas and the WSE converters (I and Q for each of 
 two polarizations), but in fact I see -2.  Is this as it 
 should be?
Yes. You have two RF channels only. You might have used the
hardware described here:
http://www.sm5bsz.com/pcdsp/hware.htm
This uses two audio channels to receive two RF channels but
the output from timf2 would have the same format as you have with
the WSE units. The difference is that the sampling speed of
timf2 is the same as the audio sampling speed with WSE units, 
but half the audio sampling speed for the other solution.
There is no need for MAP65 to know whether the 96kHz timf2 data
originates in a hardware using two audio channels with a sampling
rate of 192 kHz or four channels sampling at 96 kHz.

 Similarly, I would have expected passband_direction=1 as I 
 have no LO's on the high side and the spectrum is not 
 inverted.  However, Linrad sends passband_direction=-1.  Is 
 this correct?
You have the RX10700 with LO=13.175, 13.200, 13.225 or 13.250 MHz 
which is above 10.7. All the other LOs are below so -1 is
correct.

 Linrad FFT Versions
 ---
 
 I have been using FFT Version 0 for the first forward, first 
 backward, and second forward FFTs.  This produces 
 floating-point TIMF2 data on the multicast network.
On a modern computer you can use version 5 for the first
forward fft. It uses the SIMD instructions (single instruction
multiple data, now called sse I think) and computes the
floating point fft quite a bit faster.
  
 To save on memory usage in MAP65 it may be desirable to use 
 16-bit data for TIMF2.  I have effected this by setting 
 first forward FFT to version 5,
The first fft is always floating point. It must be because 
the dynamic range of 16 bits is by far not adequate for
the unprocessed A/D input. Version 5 is always a good choice
on a computer that will allow it. Pentium II and older do
not have the SIMD instructions so they have to use another
version and it differs a little which one will run fastest
depending on architecture. On old machines it might be desireable
to actually try all versions and pick the one that runs fastest
since old machines may be CPU limited.


 first backward FFT to 
 version 1, and second forward FFT to version 2.  Are these 
 good choices?  
Yes.

 Is there any downside to their use that I 
 might not have thought about?  Also, with these settings I 
 notice that the signal in the high resolution graph (red dB 
 lines) is higher than it was with FFT versions 0.  Should I 
 adjust some other parameter to bring this level down be 
 several dB?
When the floating point data from fft1 is truncated to 16 bits
there will be quantization noise. You should place the system
noise floor at least 20 dB above this extra source of noise to
make the loss of NF smaller than 0.043 dB.

Press A on the keyboard to see the amplitude margins. In case
you place the noise floor too high you might find that there is
saturation somewhere. The shift parameters for the FFTs will
allow you to set the noise floor at the correct level.
http://www.sm5bsz.com/linuxdsp/install/dlevel.htm

There are many possible sources of rounding errors (quantization
noise) and Linrad does not set levels automatically since the
criteria for what will be optimum are non-trivial. I hope
the above link will be helpful.

Under normal circumstances you will not need the maximum 
dynamic range in the second FFT. Only if you have a carefully
calibrated system and want to use the smart blanker saturation
on noise pulses will be a problem. Narrowband signals are
automatically attenuated to fit within the dynamic range of 
16 bits. 

In case you want a very large size for the second fft and a 
small size for the first fft, the narrowband signals have to be 
attenuated to a pretty low level. 

A perfect sinewave may put nearly all its energy in a single 
FFT bin.With a 

[linrad] Re: Using the Linrad Network

2007-06-14 Thread Joe Taylor

Leif and all,

It turns out that my network speed problem had a simple cause. 
The computer running Linrad is connected to the LAN through a 
rather old 10 Mbit/s hub/repeater.  Therefore it maxes out 
around 1.2 MByte/s, too slow for 4 channels at 96 kHz in any of 
Linrad's network modes except mode 4.


Looks like I need a new hub, or at least a simple crossover 
cable between the two computers.


I will report further, fairly soon I hope, on my effort to 
connect Linrad to MAP65.


-- Joe, K1JT

#
This message is sent to you because you are subscribed to
 the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Re: Using the Linrad Network

2007-06-13 Thread Leif Asbrink
Hello Joe,


 Today I made my first tests of Linrad's network broadcasting 
 capability.  I installed Linrad 02.34 and checked it out.  All 
 seemed to be well.  I selected N on the main menu to 
 initialize networking parameters, and then selected format 4 
 (16-bit raw data) for output.  After toggling network write ON, 
 I found that the program continued to run correctly in Weak CW mode.
Ok.

 Unfortunately, I did not have good luck with any of the other 
 data formats.  In particular, formats 7, 8, and 9 caused the 
 program to produce broken sound output -- that is, frequent gaps 
 in the Rx background noise when an active frequency is selected 
 with the mouse.
 
 Hitting T while receiving, so as to display timing 
 information, shows an apparent A/D sampling rates as follows:
 
 Format A/D
 ---
 4 96 kHz
 7~30
 8~73
 9~49
 
 Obviously, something is seriously wrong!
Yes. 

While writing this mail on my single Pentium 2.66 MHz machine,
I have one issue of Linrad running two RF channels at 96 kHz 
from the WSE units sending fft1 data to the network. I also
have two more issues of Linrad receiving the fft1 transforms.
All three issues are sending data to the loudspeakers
simultaneously. I notice no problems of any kind.
This is under Debian with kernel 2.6.18-4-686-bigmem
and oss-linux_v4.0-1002_i386.deb

When I start the system monitor I get a short break just when
the program opens. All three Linrad issues have the second FFT 
enabled and the total CPU load according to the system monitor 
is about 40% with occasional peaks up to 70%.

When I start a fourth issue of Linrad that reads 154 kHz from
an SDR-14 and also sends data (16 bit raw) to the network,
things still work fine with a total CPU load of 60% using
1.2 gigabytes of RAM. In this situation, the USB driver 
for SDR-14 crashes in case I try click on any other window
because there is not an appropriate error handeling when
data is lost from the USB device. This does not affect
the other three issues of Linrad.

It seems to me that the problems you see could be caused by
excessive RAM allocation that would cause active data areas 
of Linrad to be swapped on the hard disk. It could also
be a problem due to interrupt sharing.

I noticed a bug. In case I press T when Linrad runs from
fft1 input, the A/D rate becomes 192 kHz and an error message
is displayed. This does not affect the processing however.

 I did notice that when using Format 9, although the audio on the 
 master computer has frequent interruptions, watzo running on a 
 second computer displays waterfall data that looks more or less 
 correct.
 
 I tried to use one instance of xlinrad to broadcast format 4 
 data, and a second instance of xlinrad on the same computer 
 receiving the raw 16-bit data.  This failed with error #1272 
 (Write to network socket returned an error) on the master and 
 error #1274 (Network read failed) on the slave.
Something is wrong. Presumably the reason is the same as for
the other errors you observe.

 Can you help me to understand what may be wrong in my setup? 
 The computer on which Linrad is running is reasonably fast (2.7 
 GHz P4, 0.5 MB cache, 1.25 GB memory), and is around 20% busy 
 when running xlinrad.
Check for memory usage first.

73

Leif

#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Re: Using the Linrad Network

2007-06-13 Thread Joe Taylor

Hi Leif,

Thanks for the reply.


Something is wrong. Presumably the reason is the same as for
the other errors you observe.


  ...


Check for memory usage first.


No swapping is taking place, so I don't think it's a memory 
issue.  I will study the problem more and report back if I learn 
anything useful.


-- Joe

#
This message is sent to you because you are subscribed to
 the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Using the Linrad Network

2007-06-12 Thread Joe Taylor

He Leif and all,

Today I made my first tests of Linrad's network broadcasting 
capability.  I installed Linrad 02.34 and checked it out.  All 
seemed to be well.  I selected N on the main menu to 
initialize networking parameters, and then selected format 4 
(16-bit raw data) for output.  After toggling network write ON, 
I found that the program continued to run correctly in Weak CW mode.


Unfortunately, I did not have good luck with any of the other 
data formats.  In particular, formats 7, 8, and 9 caused the 
program to produce broken sound output -- that is, frequent gaps 
in the Rx background noise when an active frequency is selected 
with the mouse.


Hitting T while receiving, so as to display timing 
information, shows an apparent A/D sampling rates as follows:


Format A/D
---
4 96 kHz
7~30
8~73
9~49

Obviously, something is seriously wrong!

I did notice that when using Format 9, although the audio on the 
master computer has frequent interruptions, watzo running on a 
second computer displays waterfall data that looks more or less 
correct.


I tried to use one instance of xlinrad to broadcast format 4 
data, and a second instance of xlinrad on the same computer 
receiving the raw 16-bit data.  This failed with error #1272 
(Write to network socket returned an error) on the master and 
error #1274 (Network read failed) on the slave.


Can you help me to understand what may be wrong in my setup? 
The computer on which Linrad is running is reasonably fast (2.7 
GHz P4, 0.5 MB cache, 1.25 GB memory), and is around 20% busy 
when running xlinrad.


-- 73, Joe, K1JT

#
This message is sent to you because you are subscribed to
 the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Network and bits

2007-01-12 Thread Leif Asbrink
Hi all,

The network is now working fine although several things
are still missing. I have decided to abandon the idea
of using 1024 byte fixed block size because the network
in itself is too much CPU load to make it attractive on
old computers at high data rates.

As it turns out the size of the packet does not matter
at all for the CPU load and it does not matter when
two computers are tied directly to each other. When
the packets have to pass a switch it is different however.
At high rates the network does not work through the 
switch if the packet size is a little above 1500 bytes,
but it works well when the size is a little below.

Besides allowing MAP65 and other external softwares
the Linrad network is intended to encourage experimentation.
I have decided to not freeze the protocol at an early stage.
With routines that copy data from a buffer of an arbitrary 
size (it has to be a multiple of 12 however) the size
can be adjusted in case there will be future needs.
The current payload is 1404 bytes plus a 24 byte header.

One of the obvious experiments is to process the data
with different number of bits simultaneously. The
result is presented here:
http://www.sm5bsz.com/linuxdsp/run/network.htm
There is also some more info about the network on
this page.


   73

Leif / SM5BSZ

#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] AW: [linrad] Network speed problems.

2007-01-08 Thread Koellner, Guenter
Hello,

here is what we found when we had some network performance problems in the 
office:

se a well known search engine and look for the phrase net.core.rmem_default
for example described in  
http://wwwx.cs.unc.edu/~sparkst/howto/network_tuning.php

Please do not ask me any special details, I only read it and took it.

During some earlier network optimizations we also found that just looking at 
the raw amount of data beeing transfered is only partly correct. Some network 
stacks reach the limit first by the number of packets, not the amount of data. 
So it might be useful to make transport blocks as large as possible. Just like 
a Layer4, just inverse.

73, G√ľnter, DL4MEA


-Urspr√ľngliche Nachricht-
Von: Linrad mailinglist [mailto:[EMAIL PROTECTED] Im Auftrag von Leif Asbrink
Gesendet: 07 January 2007 00:28
An: Linrad mailinglist
Betreff: [linrad] Network speed problems.

Hi all,

Having been sucessful with raw data at speeds up to
96000*4*(16+18+24)=22 Mb/s I sort of believed 
it was possible to run at rates reasonably near
100 Mb/s when only sendind data in one direction
since the machine says: Link is up at 100Mbps, full duplex

Now, as it turns out I did the above test with only
one RF channel so the speed was only 11 Mb/s.
What is more of a problem is that the maximum
throughput through one socket is actually just a little
below 96000*4*16 = 6.1 MB/s. It is perfectly ok
to send 5MB/s through four sockets simultaneously
but sending fft1 transforms through a single
socket is impossible.

Presumably this is well known, I an just a newcomer
to networking and I could not guess it would behave
this way.

Is there some settings I should change? One possibillity
seems to be to open several sockets on different ports
simultaneously to increase the throughput.

I can not send even 16 bit noise-blanked time domain data
from Linrad to MAP65 over the network on a single socket.
14 bit data will be fine however because the Linrad 
processing removes all strong signals when the MMX routines
are selected.  

Is there something wrong in what I do? Does Windows behave
the same way? 

To be able to run at 96 khz with fft1 transfer it seems
I will have to use at least 6 sockets in parallel with
different port numbers.

It is interesting to note that the old Pentium MMX at 200 MHz
uses very much more CPU (under an old kernel) but that the
throughput through one socket does not differ much. It can
not send more than a 16 bit channel because there is not
enough CPU time available.

Trying to find some hints on the Internet I came across this:
http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/IBMp690/IBM/usr/share/man/info/en_US/a_doc_lib/aixbman/prftungd/2365c93.htm

* Do not change the default (and maximum) MTU of 1500 bytes.
* Set the application block size in multiples of 4096 bytes.
* Keep socket space settings at the default values.
* If the workload includes extensive use of services that use UDP, such as 
NFS or RPC, increase sb_max to allow for the fact that each 1500-byte MTU uses 
a 4096-byte buffer. 

It seems to indicate that I should send much larger packets???
(a multiple of 4096 bytes 'header' included)


  73

  Leif / SM5BSZ


#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]


#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]



[linrad] Network speed problems.

2007-01-06 Thread Leif Asbrink
Hi all,

Having been sucessful with raw data at speeds up to
96000*4*(16+18+24)=22 Mb/s I sort of believed 
it was possible to run at rates reasonably near
100 Mb/s when only sendind data in one direction
since the machine says: Link is up at 100Mbps, full duplex

Now, as it turns out I did the above test with only
one RF channel so the speed was only 11 Mb/s.
What is more of a problem is that the maximum
throughput through one socket is actually just a little
below 96000*4*16 = 6.1 MB/s. It is perfectly ok
to send 5MB/s through four sockets simultaneously
but sending fft1 transforms through a single
socket is impossible.

Presumably this is well known, I an just a newcomer
to networking and I could not guess it would behave
this way.

Is there some settings I should change? One possibillity
seems to be to open several sockets on different ports
simultaneously to increase the throughput.

I can not send even 16 bit noise-blanked time domain data
from Linrad to MAP65 over the network on a single socket.
14 bit data will be fine however because the Linrad 
processing removes all strong signals when the MMX routines
are selected.  

Is there something wrong in what I do? Does Windows behave
the same way? 

To be able to run at 96 khz with fft1 transfer it seems
I will have to use at least 6 sockets in parallel with
different port numbers.

It is interesting to note that the old Pentium MMX at 200 MHz
uses very much more CPU (under an old kernel) but that the
throughput through one socket does not differ much. It can
not send more than a 16 bit channel because there is not
enough CPU time available.

Trying to find some hints on the Internet I came across this:
http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/IBMp690/IBM/usr/share/man/info/en_US/a_doc_lib/aixbman/prftungd/2365c93.htm

* Do not change the default (and maximum) MTU of 1500 bytes.
* Set the application block size in multiples of 4096 bytes.
* Keep socket space settings at the default values.
* If the workload includes extensive use of services that use UDP, such as 
NFS or RPC, increase sb_max to allow for the fact that each 1500-byte MTU uses 
a 4096-byte buffer. 

It seems to indicate that I should send much larger packets???
(a multiple of 4096 bytes 'header' included)


  73

  Leif / SM5BSZ


#
This message is sent to you because you are subscribed to
  the mailing list linrad@antennspecialisten.se.
To unsubscribe, E-mail to: [EMAIL PROTECTED]
To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED]
To switch to the INDEX mode, E-mail to [EMAIL PROTECTED]
Send administrative queries to  [EMAIL PROTECTED]