Hi Brock,
It appears that the second log you show is missing the middle. We don't
log a 'HOSTS' entry, so I believe that 'S' is the end of NBYTES.
However, I'm not aware of a bug like that or how that might happen. How
often do you see these corrupted or incomplete entries? Do you detect
any pattern, such as the timestamps of the nearby entries being the
same, or the pid changing right before or after? Can you show the
server config you're using?
Mike
On 9/11/2014 2:54 PM, Brock Palen wrote:
Ok now I am really confused,
I have Transfer stats entries with:
START DATE NBYTES
DATE NBYTES
DATE
Why are there three types and why are some not complete information?
Im tracking the performance of our transfers, NYBTES/(DATE - START) but when
they are not there kinda hard todo.
We are using the package:
globus-gridftp-server-6.43-1gt.x86_64
Part of the globusonline service.
Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
[email protected]
(734)936-1985
On Sep 11, 2014, at 3:17 PM, Brock Palen <[email protected]> wrote:
I made a logstash parser for transfer logs, but ran into issues because it
appears that that there are Transfer stats
Entries that do not have full information:
Eg:
What I expect:
[15298] Wed Jul 16 11:08:42 2014 :: Transfer stats: DATE=20140716150842.633689
HOST=flux-xfer1.engin.umich.edu PROG=globus-gridftp-server NL.EVNT=FTP_INFO
START=20140716150841.618581 USER=brockp FILE=/tmp/brockp/linux_x64_64_sfx.exe
BUFFER=87380 BLOCK=262144 NBYTES=280590752 VOLUME=/ STREAMS=4 STRIPES=1
DEST=[141.212.30.10] TYPE=RETR CODE=226
What I find and choke on:
[22304] Mon Sep 8 16:45:52 2014 :: Transfer stats: DATE=20140908204552.177237
HOSTS=1944 VOLUME=/ STREAMS=2 STRIPES=1 DEST=[141.212.30.10] TYPE=STOR CODE=226
What are the meanings of these two different formats? Is the first one, with
the transfer size etc, the final summary of the transfer? Can I just ignore
the others if I want to collect statistics on our gridftp servers?
Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
[email protected]
(734)936-1985
