Hi list.
Firstly, sorry for don't respond the emails, today I had a very busy day :S.
Well, I see that this problem its hard to identify the solution :P.
Tomorrow (here, in Spain, it's night already :P) I'll try upload the logs of
VenueServers on Ubuntu (with problems) and Windows (with success) system,
therefore, I'll try upload the logs of Ubuntu machine when a file is uploaded
without problems, to see the diference, on Ubuntu, in both cases.
I hope this problem fix soon without headaches jejeje.
Regards ! see you tomorrow ;)
El dia 27 abr 2010 23:11, Christoph Willing <c.will...@uq.edu.au> escribió:
On 28/04/2010, at 6:38 AM, Vine, Derek A wrote:
At least in my case, I connect to the venue locally from the venue
server, I can upload files like expected. Connecting from any other node will
not allow upload of files, at least not "live". Sometimes after you try to
upload a file and leave and come back the file is there.
Derek,
That looks like a network problem then, like when I was testing from home
and small file uploads were OK but large ones didn't seem to work - it was just
that big files took a lot longer to via the relatively slow ADSL upload
bandwidth.
All,
For comparison with previous log output, here's the server log output of a
successful upload:
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:338 INFO Connection
from 124.171.171.232:53430
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 220 vv3
M2Crypto (Medusa) FTP/TLS server v0.18 ready.
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== AUTH TLS
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 234
AUTH TLS successful
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== PBSZ 0
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 200
PBSZ=0 successful.
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== PROT P
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 200
Protection set to Private
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== USER
0a0101d80c2020bf185599cb7beea8bb
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 331
Password required.
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== PASS
<password>
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:432 INFO in
authorizeDataTransferCB <AccessGrid.FTPS.ftps_server.ftp_tls_channel connected
124.171.171.232:53430 at 0x9c6494c> 0a0101d80c2020bf185599cb7be
ea8bb 82664eb24cd2201b64c9ef4c5e7f98fe
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:458 INFO
authorizeDataTransferCB: user in venue, authorized Jaunty64 Test to transfer
files to venue Test Room
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 230 Ok.
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:275 INFO Successful
login: Filesystem=<unix-style fs root:Data wd:/>
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== cwd
/0a0101d80c2020bf185599cb7beea8bb
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 250 CWD
command successful.
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== TYPE I
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 200
Type set to Binary.
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== PASV
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 227
Entering Passive Mode (130,102,78,178,195,89)
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO <== stor
killall.diff
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 150
Opening Binary connection for killall.diff
04/28/10 07:00:08 -1245000816 FTPSServer ftps_server.py:271 INFO ==> 226
Transfer complete.
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:468 DEBUG
dataActivityCB: command = RECV
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:469 INFO
dataActivityCB: command = RECV
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:470 INFO
dataActivityCB: line = Data/0a0101d80c2020bf185599cb7beea8bb/ killall.diff
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:474 DEBUG
dataActivityCB: Pathpart is: Data
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:474 DEBUG
dataActivityCB: Pathpart is: 0a0101d80c2020bf185599cb7beea8bb
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:474 DEBUG
dataActivityCB: Pathpart is: killall.diff
04/28/10 07:00:08 -1245000816 VenueServer VenueServer.py:486 DEBUG
dataActivityCB: VenueID 0a0101d80c2020bf185599cb7beea8bb ; Filename:
killall.diff
-----Original Message-----
From: ag-tech-boun...@lists.mcs.anl.gov
[mailto:ag-tech-boun...@lists.mcs.anl.gov ] On Behalf Of Christoph Willing
Sent: Tuesday, April 27, 2010 3:27 PM
To: Vine, Derek A
Cc: ag-t...@mcs.anl.gov
Subject: Re: [AG-TECH] Problems uploading files
On 28/04/2010, at 1:31 AM, Vine, Derek A wrote:
would help if I attached the file...
________________________________________
From: ag-tech-boun...@lists.mcs.anl.gov
[ag-tech-boun...@lists.mcs.anl.gov
] On Behalf Of Vine, Derek A [derek.v...@usd.edu]
Sent: Tuesday, April 27, 2010 10:23 AM
To: Thomas Uram; Christoph Willing
Cc: ag-t...@mcs.anl.gov
Subject: Re: [AG-TECH] Problems uploading files
I am not sure if it is the same thing, but I too am having issues
uploading files to our venue server, attached is the log that I sent
to the list before but never heard anything back. My setup is
running on Windows server 2003.
Derek,
I wondered whether they were connected as I was looking around for
similar cases but assumed they were different since you mentioned
Windows - the OP in this thread said that Windows was OK for them.
However looking at your log file, I see that its almost identical to
the excerpt I just sent to the list a few minutes ago. The one
difference that I see is that you have an additional line:
02/03/10 09:33:04 4180 FTPSServer ftps_server.py:275 INFO
recv: closing channel <AccessGrid.FTPS.ftps_server.ftp_tls_channel
connected 206.176.1.99:1119 at 0x2cc23c8> (0, 'Error')
I don't have any recognition that there's an error after "INFO ==>
227 Entering Passive Mode".
chris
________________________________________
From: ag-tech-boun...@lists.mcs.anl.gov
[ag-tech-boun...@lists.mcs.anl.gov
] On Behalf Of Thomas Uram [tu...@mcs.anl.gov]
Sent: Tuesday, April 27, 2010 10:06 AM
To: Christoph Willing
Cc: ag-t...@mcs.anl.gov
Subject: Re: [AG-TECH] Problems uploading files
The VenueClient.log shows no evidence of a problem. I suspect a
server-side failure which prevents transmission of the ADD_DATA
event, which would explain the client behavior.
Has anyone captured the problem in a VenueServer.log?
Tom
On Apr 27, 2010, at 6:42 AM, Christoph Willing wrote:
On 27/04/2010, at 2:32 PM, John I. Quebedeaux, Jr wrote:
Hmmm... And ditto to Fedora 12, and 'ding' I may be having the
same problem.
When I look for the files, they appear to be '0' sized though...
But I
haven't tried a 'small' one yet.
Thanks for adding to the evidence John.
It seems there are two aspects to this problem. On the server side,
the file is not received resulting in a zero size file. However I
don't anymore think this is a data size issue - it just seemed that
way to me at first. The AG's FTPSServer prepares to start but no
data at all is actually exchanged between the client and the server.
There is a client side problem too, that even when a file has
uploaded correctly it may not appear in the venue client ui.
Its a very weird problem. I spent most of today with an "old"
system (in which server & client sides both work perfectly) and
gradually built & installed updated packages so that they all match
a current non-working system - figuring that the fault would
manifest after one of these updates, identifying it as the culprit.
"Unfortunately" the system still works at the end of all that - it
seems none of the prime suspect packages is causing the problem.
Tomorrow I'll either have to come with a new strategy or new
suspect support packages. Does anyone want to suspect, say, openssl
or even python itself?
chris
From: Christoph Willing <c.will...@uq.edu.au>
Date: Tue, 27 Apr 2010 10:04:58 +1000
To: Jesus Cea Oliva <jesus....@uca.es>
Cc: <ag-t...@mcs.anl.gov>
Subject: Re: [AG-TECH] Problems uploading files
Jesus,
Could you submit a bug report for this problem please? You
can do
that
via the VenueClient Help menu.
I have now been able to replicate the problem on several
systems -
Ubuntu, Slackware, Fedora & Debian. Interestingly older
versions
running on these systems seem to work OK i.e. when a venue
server
runs
on the older systems, the problem is not evident.
Unfortunately I
don't have resources to test all Linux systems but so far,
the last
known versions to work correctly (a client can upload to a
server
running on the same system) are:
Ubuntu Intrepid
Slackware 12.2
Fedora - unknown exactly but the problem is evident in Fedora
11
Debian - Lenny partially works
The Debian case is interesting because a Lenny client will not
upload
data to a Lenny server, yet a Jaunty client can upload to a
Lenny
server (even though it can't upload to a Jaunty server).
At this stage, I can't determine whether the problem is due
to AG
toolkit code or to one of the supporting software packages;
particular
versions of zsi and m2crypto have caused most heartburn in the
past -
maybe they're implicated again. I think it will take some
time to
find
and fix
chris
On 26/04/2010, at 12:51 AM, Jesus Cea Oliva wrote:
Hi Christoph, thanks for your indications :).
I tried local VenueServers on machine, I mean, if I run
Ubuntu, I
execute in the same machine a local VenueServer, then, I
run
VenueClient and I try upload a file, the same with Windows
7.
However, due to your mail, I've doing some tests:
- On my Ubuntu system, I've entered at
https://vv3.ap-accessgrid.org:8000/Venues/default
, the Test Room, and I download files:
VenueThermoClient.agpkg3
and
methanol.obj, and rename it adding X (metahnol.objX). Then
I can
upload the files without problems to the server.
I create a local network with 2 machines, with Windows 7
and
Ubuntu.
- Firstly I've executed a VenueServer on Ubuntu machine,
then I've
run VenueClients in both machines. I can't upload files in
the 2
machines (well you know, really, I can upload, but the
Client
appears stop, reentering in the Venue I can see the files
uploaded).
I've noticed other problem, when I try delete a file,
VenueClient
don't refresh this command, the system don't delete the
file.
- I've executed a VenueServer on Windows 7 machine, then
I've run
VenueClients in both machines. Here, I can upload files
without
problems, on Windows and Ubuntu systems.
So, now, I know that I have problems with our VenueServer
on
Ubuntu
systems, I don't know if I have to configure something or
if there
are problems about names, like ñ, spaces, etc (user name,
path
file,
etc, although my home path is /home/ercea and, there, is
the
VenueServer's Data folder)
Now I'll try change the system language to US and I'll
repeat the
tests.
Thanks again and apologize for any inconveniences
Regards!
El dia 25 abr 2010 12:11, Christoph Willing
<c.will...@uq.edu.au>
escribió:
On 23/04/2010, at 6:24 PM, Jesus Cea Oliva wrote:
Hi Crhistoph, thanks for your response :)
I try upload files since 1 o 2 K (source codes, txts,
etc.) and
files about 3-6 Mb (PDFs, ppts...) and, all cases I have
the same
problem. The strange of this problem is that ocurrs
randomly, and
sometimes, the same file that before appears not upload,
now
upload
(although, the majority cases, don't upload)
Are you using the same venue server in each test case
(Ubuntu &
Windows VenueClient)?
Are you using your own server or a "well known" server?
If these tests are with you own server, could you instead
use the
APAG venue server at
https://vv3.ap-accessgrid.org:8000/Venues/
default and then enter the Test venue. There are several
files in
that venue; could you try to download the simple.py and
methanol.obj
files from that venue and rename them e.g. simple.py ->
simple.pyX
and methanol.obj -> methanol.objX. Now upload te renamed
files
back
into the Test venue. Do they both upload as expected?
FYI, although I thought I had replicated the problem, I
now find
that it actually works correctly. My first hurried tests
only
appeared to replicate your symptoms - because I didn't
wait long
enogh for the upload to complete. However I have now
performed the
tests again and all files upload correctly - and the venue
client
interface shows the new data correctly in all cases.
I believe the previous test appeared not to update the
venue
client
interface because I was performing the tests from home
using an
ADSL
connection - therefore the initial download was quite
fast. The
upload was slower because of the much lower upstream
bandwidth
of my
ADSL connection. My guess is that the first portion of
upload
_appears_ fast because the venue client is just filling up
the
local
TCP system buffer. However that buffer then has to be
transferred
(via slow ADSL uplink) to the venue server - which
manifests as a
fast intitial movement of the progress bar that then
appears to
freeze. The "freeze" is really just a pause while contents
of the
TCP buffer is being transferred across the network.
In my case the simple.py file, being small, transfers very
quickly.
The methanol.obj file is about 11M - I guess this is just
a bit
larger than my system network buffer size - and takes a
lot longer
to transfer.
Based on redoing the tests (using the APAG veneu server),
I cannot
replicate the problem.
Could you confirm whether you still have the problem when
you use
the APAG venue server?
chris
On Windows I try similar files (somes files are the same),
only
don't work once due to a very big file. As I saw that I
can upload
files without problems... I try all file types. So I
upload a file
about 100Mb and don't work xD.
Thanks for your help :). Regards!
El dia 23 abr 2010 02:35, Christoph Willing
<c.will...@uq.edu.au>
escribió:
I wonder if this is related to the size of the data
object. I just
tried (with Ubuntu 9.10 64bit) and found that a small file
(6K)
there was no problem. With a larger file (12M) I saw
exactly the
same problem - upload appears not to complete but has
actually
succeeded. Re-entering the venue shows that the new data
is there.
In your Windows test, are you uploading the same file (or
something
of similar size) as in the Ubuntu case?
I will now be away for a few days so I can't do any more
investigation myself until I return. Maybe others can look
into
this
in the meantime ...
chris
On 23/04/2010, at 5:28 AM, Jesus Cea Oliva wrote:
Thanks Tom for your fast response :).
Well, atach the VenueClient log in this email, I can't see
any
problems :S.
04/22/10 21:16:50 140471913264880 VenueClient
VenueClientUI.py:
1308
DEBUG VenueClientUI.AddDataCB: Trying to upload to
'ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007'
04/22/10 21:17:00 140471913264880 VenueClient
VenueClientUI.py:
1395
DEBUG AddDataCB: URI of parent is
04/22/10 21:17:00 140471913264880 VenueClientController
VenueClientController.py:821 DEBUG In
VenueClientController.UploadVenueFiles
04/22/10 21:17:00 140471913264880 VenueClientController
VenueClientController.py:822 DEBUG fileList =
[u'/home/ercea/
Descargas/DSD-Tutorial-3.pdf']
04/22/10 21:17:00 140471913264880 VenueClientController
VenueClientController.py:848 DEBUG Serverpath:
04/22/10 21:17:00 140471913264880 VenueClientController
VenueClientController.py:857 DEBUG Have args, creating
thread,
url:
ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
, files: [u'/home/ercea/Descargas/DSD-Tutorial-3.pdf']
04/22/10 21:17:00 140471479552272 VenueClientController
VenueClientController.py:944 DEBUG Upload: getting identity
04/22/10 21:17:00 140471479552272 VenueClientController
VenueClientController.py:949 DEBUG Got identity <?xml
version="1.0" ? ><Subject><Subject auth_data=""
auth_type="x509"
name="/O=Access Grid/O=Argonne National
Laboratory/OU=Futures Lab
Anonymous Authority/CN=Anonymous User
17130398eef9fd82cba8dc9d845a1768"/></ Subject>
04/22/10 21:17:00 140471479552272 VenueClientController
VenueClientController.py:950 DEBUG get_ident_and_upload:
Upload
URL
ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
04/22/10 21:17:00 140471479552272 VenueClientController
VenueClientController.py:953 DEBUG Get_ident_and_upload:
Word is:
ftps:
04/22/10 21:17:00 140471479552272 VenueClientController
VenueClientController.py:953 DEBUG Get_ident_and_upload:
Word is:
04/22/10 21:17:00 140471479552272 VenueClientController
VenueClientController.py:953 DEBUG Get_ident_and_upload:
Word is:
localhost:8006
04/22/10 21:17:00 140471479552272 VenueClientController
VenueClientController.py:953 DEBUG Get_ident_and_upload:
Word is:
7f00010133381d7afaba1c148fcabe1007
04/22/10 21:17:00 140471479552272 DataStore
DataStore.py:1267 INFO
UploadFiles: ftps://localhost:
8006/7f00010133381d7afaba1c148fcabe1007
[u'/home/ercea/Descargas/
DSD- Tutorial-3.pdf']
04/22/10 21:17:00 140471479552272 DataStore
DataStore.py:1271
DEBUG
UploadFiles: ftps://localhost:
8006/7f00010133381d7afaba1c148fcabe1007
/home/ercea/Descargas/ DSD-
Tutorial-3.pdf [u'/home/ercea/Descargas/
DSD-Tutorial-3.pdf']
04/22/10 21:17:00 140471479552272 FTPSClient
FTPSClient.py:141
DEBUG
Entered FTPSUploadFile:
localfile=/home/ercea/Descargas/DSD-
Tutorial-3.pdf
url=ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
04/22/10 21:17:00 140471913264880 VenueClientController
VenueClientController.py:862 DEBUG Started thread
04/22/10 21:17:20 140471517653264 VenueClient
VenueClient.py:605
DEBUG Calling Heartbeat, time now: 1271963840
04/22/10 21:17:20 140471517653264 VenueClient
VenueClient.py:629
DEBUG Next Heartbeat needed within 36s
04/22/10 21:17:20 140471517653264 VenueClient
VenueClient.py:639
DEBUG heartBeatCounter = 2
04/22/10 21:17:56 140471401707792 VenueClient
VenueClient.py:605
DEBUG Calling Heartbeat, time now: 1271963876
04/22/10 21:17:56 140471401707792 VenueClient
VenueClient.py:629
DEBUG Next Heartbeat needed within 36s
04/22/10 21:17:56 140471401707792 VenueClient
VenueClient.py:639
DEBUG heartBeatCounter = 3
Here increment heartBeatCounter +1.
Ah, I have this problem, in the same network, on ubuntu
system
(Ubuntu 9.10 64 bits), but I've tested on Windows 7 and I
can
upload
files without problems.
Regards!
El dia 22 abr 2010 18:29, Thomas Uram <tu...@mcs.anl.gov>
escribió:
Hi Jesus:
There is probably some indication of the problem in the
VenueClient.log file. If you submit a bug report from the
Help
menu
after this happens, a recent portion of the
VenueClient.log file
will be included with the report, and we can review it. If
you
want
to look yourself, the log file resides in the following
locations
based on platform:
MacOS, Linux: ~/.AccessGrid3/Logs
Windows: c:\documents and settings\username\application
data
\AccessGrid3\Logs (adjusted for your locale, of course)
Tom
On Apr 22, 2010, at 11:11 AM, Jesus Cea Oliva wrote:
Hi list again, sorry for my constants problems xD.
I'm trying upload files to our VenueServer and my problem
is
strange. Firstly, I've created a VenueServer locally, for
testing.
Well, when I try to upload a file, I can see the message
on the
VenueClient "Uploading file X" and a progressbar, but,
when the
progressbar is near to the end, stops and list of files
(Data
section) don't refresh. So I beleived that the file can't
upload.
But, really, the file has uploaded, because I can see the
folder
Data of my local VenueServer, and the file is copied in
this
folder.
In this case, if I reset VenueServer and VenueClient,
then, I can
see the files in Data section.
So I think that the file uploads fine but, the list of
files don't
refresh and the file uploaded don't appears.
I must say that, sometimes, I can upload a file so I don't
know
whats really the problem (ports closed? firewall? path or
name
files? file size?) .
Thanks again and apologize for any inconveniences
Regards!
<VenueClient.log>
Christoph Willing +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland
Christoph Willing +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland
Christoph Willing +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland
Christoph Willing +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland
<USD Venue Server log.txt>
Christoph Willing +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland
Christoph Willing +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland
