On 28/04/2010, at 1:06 AM, Thomas Uram wrote: > The VenueClient.log shows no evidence of a problem. I suspect a > server-side failure which prevents transmission of the ADD_DATA > event, which would explain the client behavior. > > Has anyone captured the problem in a VenueServer.log?
Tom, Here's all the FTPS related stuff from a server log file. In this case, 130.102.78.169 is running both venue server and client. "killall.diff" is the file being uploaded (428 bytes), "Slackware-13.0 Test" is the name of the venue client user. In the server's Data directory for that venue, the file is created but it remains empty. chris 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:338 INFO Connection from 130.102.78.169:2150 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 220 v2 M2Crypto (Medusa) FTP/TLS server v0.20.2 ready. 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== AUTH TLS 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 234 AUTH TLS successful 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== PBSZ 0 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 200 PBSZ=0 successful. 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== PROT P 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 200 Protection set to Private 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== USER 82664ea968181fef2e9442429c415853 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 331 Password required. 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== PASS <password> 04/27/10 22:59:20 -1245144176 VenueServer VenueServer.py:433 INFO in authorizeDataTransferCB <AccessGrid.FTPS.ftps_server.ftp_tls_channel connected 130.102.78.169:2150 at 0x91cdbec> 82664ea968181fef2e9442429c415853 82664ea968181fef422234f4f84c03f3 04/27/10 22:59:20 -1245144176 VenueServer VenueServer.py:459 INFO authorizeDataTransferCB: user in venue, authorized Slackware-13.0 Test to transfer files to venue Venue Server Lobby 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 230 Ok. 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:275 INFO Successful login: Filesystem=<unix-style fs root:Data wd:/> 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== cwd /82664ea968181fef2e9442429c415853 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 250 CWD command successful. 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== TYPE I 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 200 Type set to Binary. 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== PASV 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 227 Entering Passive Mode (130,102,78,169,195,90) 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO <== stor killall.diff 04/27/10 22:59:20 -1245144176 FTPSServer ftps_server.py:271 INFO ==> 150 Opening Binary connection for killall.diff > On Apr 27, 2010, at 6:42 AM, Christoph Willing wrote: > >> >> On 27/04/2010, at 2:32 PM, John I. Quebedeaux, Jr wrote: >> >>> Hmmm... And ditto to Fedora 12, and 'ding' I may be having the >>> same problem. >>> When I look for the files, they appear to be '0' sized though... >>> But I >>> haven't tried a 'small' one yet. >> >> >> Thanks for adding to the evidence John. >> >> It seems there are two aspects to this problem. On the server side, >> the file is not received resulting in a zero size file. However I >> don't anymore think this is a data size issue - it just seemed that >> way to me at first. The AG's FTPSServer prepares to start but no >> data at all is actually exchanged between the client and the server. >> >> There is a client side problem too, that even when a file has >> uploaded correctly it may not appear in the venue client ui. >> >> Its a very weird problem. I spent most of today with an "old" >> system (in which server & client sides both work perfectly) and >> gradually built & installed updated packages so that they all match >> a current non-working system - figuring that the fault would >> manifest after one of these updates, identifying it as the culprit. >> "Unfortunately" the system still works at the end of all that - it >> seems none of the prime suspect packages is causing the problem. >> Tomorrow I'll either have to come with a new strategy or new >> suspect support packages. Does anyone want to suspect, say, openssl >> or even python itself? >> >> >> chris >> >> >>>> From: Christoph Willing <c.will...@uq.edu.au> >>>> Date: Tue, 27 Apr 2010 10:04:58 +1000 >>>> To: Jesus Cea Oliva <jesus....@uca.es> >>>> Cc: <ag-t...@mcs.anl.gov> >>>> Subject: Re: [AG-TECH] Problems uploading files >>>> >>>> Jesus, >>>> >>>> Could you submit a bug report for this problem please? You can do >>>> that >>>> via the VenueClient Help menu. >>>> >>>> >>>> I have now been able to replicate the problem on several systems - >>>> Ubuntu, Slackware, Fedora & Debian. Interestingly older versions >>>> running on these systems seem to work OK i.e. when a venue server >>>> runs >>>> on the older systems, the problem is not evident. Unfortunately I >>>> don't have resources to test all Linux systems but so far, the last >>>> known versions to work correctly (a client can upload to a server >>>> running on the same system) are: >>>> Ubuntu Intrepid >>>> Slackware 12.2 >>>> Fedora - unknown exactly but the problem is evident in Fedora 11 >>>> Debian - Lenny partially works >>>> >>>> The Debian case is interesting because a Lenny client will not >>>> upload >>>> data to a Lenny server, yet a Jaunty client can upload to a Lenny >>>> server (even though it can't upload to a Jaunty server). >>>> >>>> >>>> At this stage, I can't determine whether the problem is due to AG >>>> toolkit code or to one of the supporting software packages; >>>> particular >>>> versions of zsi and m2crypto have caused most heartburn in the >>>> past - >>>> maybe they're implicated again. I think it will take some time to >>>> find >>>> and fix >>>> >>>> >>>> chris >>>> >>>> >>>> On 26/04/2010, at 12:51 AM, Jesus Cea Oliva wrote: >>>> >>>>> Hi Christoph, thanks for your indications :). >>>>> >>>>> I tried local VenueServers on machine, I mean, if I run Ubuntu, I >>>>> execute in the same machine a local VenueServer, then, I run >>>>> VenueClient and I try upload a file, the same with Windows 7. >>>>> >>>>> However, due to your mail, I've doing some tests: >>>>> >>>>> - On my Ubuntu system, I've entered at >>>>> https://vv3.ap-accessgrid.org:8000/Venues/default >>>>> , the Test Room, and I download files: VenueThermoClient.agpkg3 >>>>> and >>>>> methanol.obj, and rename it adding X (metahnol.objX). Then I can >>>>> upload the files without problems to the server. >>>>> >>>>> I create a local network with 2 machines, with Windows 7 and >>>>> Ubuntu. >>>>> >>>>> - Firstly I've executed a VenueServer on Ubuntu machine, then I've >>>>> run VenueClients in both machines. I can't upload files in the 2 >>>>> machines (well you know, really, I can upload, but the Client >>>>> appears stop, reentering in the Venue I can see the files >>>>> uploaded). >>>>> I've noticed other problem, when I try delete a file, VenueClient >>>>> don't refresh this command, the system don't delete the file. >>>>> >>>>> >>>>> - I've executed a VenueServer on Windows 7 machine, then I've run >>>>> VenueClients in both machines. Here, I can upload files without >>>>> problems, on Windows and Ubuntu systems. >>>>> >>>>> So, now, I know that I have problems with our VenueServer on >>>>> Ubuntu >>>>> systems, I don't know if I have to configure something or if there >>>>> are problems about names, like ñ, spaces, etc (user name, path >>>>> file, >>>>> etc, although my home path is /home/ercea and, there, is the >>>>> VenueServer's Data folder) >>>>> >>>>> Now I'll try change the system language to US and I'll repeat the >>>>> tests. >>>>> >>>>> >>>>> Thanks again and apologize for any inconveniences >>>>> Regards! >>>>> >>>>> >>>>> >>>>> El dia 25 abr 2010 12:11, Christoph Willing <c.will...@uq.edu.au> >>>>> escribió: >>>>> On 23/04/2010, at 6:24 PM, Jesus Cea Oliva wrote: >>>>> >>>>> Hi Crhistoph, thanks for your response :) >>>>> >>>>> I try upload files since 1 o 2 K (source codes, txts, etc.) and >>>>> files about 3-6 Mb (PDFs, ppts...) and, all cases I have the same >>>>> problem. The strange of this problem is that ocurrs randomly, and >>>>> sometimes, the same file that before appears not upload, now >>>>> upload >>>>> (although, the majority cases, don't upload) >>>>> >>>>> Are you using the same venue server in each test case (Ubuntu & >>>>> Windows VenueClient)? >>>>> >>>>> >>>>> Are you using your own server or a "well known" server? >>>>> If these tests are with you own server, could you instead use the >>>>> APAG venue server at https://vv3.ap-accessgrid.org:8000/Venues/ >>>>> default and then enter the Test venue. There are several files in >>>>> that venue; could you try to download the simple.py and >>>>> methanol.obj >>>>> files from that venue and rename them e.g. simple.py -> simple.pyX >>>>> and methanol.obj -> methanol.objX. Now upload te renamed files >>>>> back >>>>> into the Test venue. Do they both upload as expected? >>>>> >>>>> >>>>> FYI, although I thought I had replicated the problem, I now find >>>>> that it actually works correctly. My first hurried tests only >>>>> appeared to replicate your symptoms - because I didn't wait long >>>>> enogh for the upload to complete. However I have now performed the >>>>> tests again and all files upload correctly - and the venue client >>>>> interface shows the new data correctly in all cases. >>>>> >>>>> I believe the previous test appeared not to update the venue >>>>> client >>>>> interface because I was performing the tests from home using an >>>>> ADSL >>>>> connection - therefore the initial download was quite fast. The >>>>> upload was slower because of the much lower upstream bandwidth >>>>> of my >>>>> ADSL connection. My guess is that the first portion of upload >>>>> _appears_ fast because the venue client is just filling up the >>>>> local >>>>> TCP system buffer. However that buffer then has to be transferred >>>>> (via slow ADSL uplink) to the venue server - which manifests as a >>>>> fast intitial movement of the progress bar that then appears to >>>>> freeze. The "freeze" is really just a pause while contents of the >>>>> TCP buffer is being transferred across the network. >>>>> >>>>> In my case the simple.py file, being small, transfers very >>>>> quickly. >>>>> The methanol.obj file is about 11M - I guess this is just a bit >>>>> larger than my system network buffer size - and takes a lot longer >>>>> to transfer. >>>>> >>>>> Based on redoing the tests (using the APAG veneu server), I cannot >>>>> replicate the problem. >>>>> >>>>> Could you confirm whether you still have the problem when you use >>>>> the APAG venue server? >>>>> >>>>> >>>>> chris >>>>> >>>>> >>>>> On Windows I try similar files (somes files are the same), only >>>>> don't work once due to a very big file. As I saw that I can upload >>>>> files without problems... I try all file types. So I upload a file >>>>> about 100Mb and don't work xD. >>>>> >>>>> Thanks for your help :). Regards! >>>>> >>>>> El dia 23 abr 2010 02:35, Christoph Willing <c.will...@uq.edu.au> >>>>> escribió: >>>>> I wonder if this is related to the size of the data object. I just >>>>> tried (with Ubuntu 9.10 64bit) and found that a small file (6K) >>>>> there was no problem. With a larger file (12M) I saw exactly the >>>>> same problem - upload appears not to complete but has actually >>>>> succeeded. Re-entering the venue shows that the new data is there. >>>>> >>>>> In your Windows test, are you uploading the same file (or >>>>> something >>>>> of similar size) as in the Ubuntu case? >>>>> >>>>> I will now be away for a few days so I can't do any more >>>>> investigation myself until I return. Maybe others can look into >>>>> this >>>>> in the meantime ... >>>>> >>>>> >>>>> chris >>>>> >>>>> >>>>> On 23/04/2010, at 5:28 AM, Jesus Cea Oliva wrote: >>>>> >>>>> Thanks Tom for your fast response :). >>>>> >>>>> Well, atach the VenueClient log in this email, I can't see any >>>>> problems :S. >>>>> >>>>> >>>>> >>>>> 04/22/10 21:16:50 140471913264880 VenueClient VenueClientUI.py: >>>>> 1308 >>>>> DEBUG VenueClientUI.AddDataCB: Trying to upload to >>>>> 'ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007' >>>>> 04/22/10 21:17:00 140471913264880 VenueClient VenueClientUI.py: >>>>> 1395 >>>>> DEBUG AddDataCB: URI of parent is >>>>> 04/22/10 21:17:00 140471913264880 VenueClientController >>>>> VenueClientController.py:821 DEBUG In >>>>> VenueClientController.UploadVenueFiles >>>>> 04/22/10 21:17:00 140471913264880 VenueClientController >>>>> VenueClientController.py:822 DEBUG fileList = [u'/home/ercea/ >>>>> Descargas/DSD-Tutorial-3.pdf'] >>>>> 04/22/10 21:17:00 140471913264880 VenueClientController >>>>> VenueClientController.py:848 DEBUG Serverpath: >>>>> 04/22/10 21:17:00 140471913264880 VenueClientController >>>>> VenueClientController.py:857 DEBUG Have args, creating thread, >>>>> url: >>>>> ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007 >>>>> , files: [u'/home/ercea/Descargas/DSD-Tutorial-3.pdf'] >>>>> 04/22/10 21:17:00 140471479552272 VenueClientController >>>>> VenueClientController.py:944 DEBUG Upload: getting identity >>>>> 04/22/10 21:17:00 140471479552272 VenueClientController >>>>> VenueClientController.py:949 DEBUG Got identity <?xml >>>>> version="1.0" ? ><Subject><Subject auth_data="" auth_type="x509" >>>>> name="/O=Access Grid/O=Argonne National Laboratory/OU=Futures Lab >>>>> Anonymous Authority/CN=Anonymous User >>>>> 17130398eef9fd82cba8dc9d845a1768"/></ Subject> >>>>> 04/22/10 21:17:00 140471479552272 VenueClientController >>>>> VenueClientController.py:950 DEBUG get_ident_and_upload: Upload >>>>> URL >>>>> ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007 >>>>> 04/22/10 21:17:00 140471479552272 VenueClientController >>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is: >>>>> ftps: >>>>> 04/22/10 21:17:00 140471479552272 VenueClientController >>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is: >>>>> 04/22/10 21:17:00 140471479552272 VenueClientController >>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is: >>>>> localhost:8006 >>>>> 04/22/10 21:17:00 140471479552272 VenueClientController >>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is: >>>>> 7f00010133381d7afaba1c148fcabe1007 >>>>> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1267 INFO >>>>> UploadFiles: ftps://localhost: >>>>> 8006/7f00010133381d7afaba1c148fcabe1007 [u'/home/ercea/Descargas/ >>>>> DSD- Tutorial-3.pdf'] >>>>> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1271 >>>>> DEBUG >>>>> UploadFiles: ftps://localhost: >>>>> 8006/7f00010133381d7afaba1c148fcabe1007 /home/ercea/Descargas/DSD- >>>>> Tutorial-3.pdf [u'/home/ercea/Descargas/ DSD-Tutorial-3.pdf'] >>>>> 04/22/10 21:17:00 140471479552272 FTPSClient FTPSClient.py:141 >>>>> DEBUG >>>>> Entered FTPSUploadFile: localfile=/home/ercea/Descargas/DSD- >>>>> Tutorial-3.pdf >>>>> url=ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007 >>>>> 04/22/10 21:17:00 140471913264880 VenueClientController >>>>> VenueClientController.py:862 DEBUG Started thread >>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:605 >>>>> DEBUG Calling Heartbeat, time now: 1271963840 >>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:629 >>>>> DEBUG Next Heartbeat needed within 36s >>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:639 >>>>> DEBUG heartBeatCounter = 2 >>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:605 >>>>> DEBUG Calling Heartbeat, time now: 1271963876 >>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:629 >>>>> DEBUG Next Heartbeat needed within 36s >>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:639 >>>>> DEBUG heartBeatCounter = 3 >>>>> >>>>> >>>>> Here increment heartBeatCounter +1. >>>>> >>>>> Ah, I have this problem, in the same network, on ubuntu system >>>>> (Ubuntu 9.10 64 bits), but I've tested on Windows 7 and I can >>>>> upload >>>>> files without problems. >>>>> >>>>> Regards! >>>>> >>>>> >>>>> El dia 22 abr 2010 18:29, Thomas Uram <tu...@mcs.anl.gov> >>>>> escribió: >>>>> Hi Jesus: >>>>> >>>>> There is probably some indication of the problem in the >>>>> VenueClient.log file. If you submit a bug report from the Help >>>>> menu >>>>> after this happens, a recent portion of the VenueClient.log file >>>>> will be included with the report, and we can review it. If you >>>>> want >>>>> to look yourself, the log file resides in the following locations >>>>> based on platform: >>>>> MacOS, Linux: ~/.AccessGrid3/Logs >>>>> Windows: c:\documents and settings\username\application data >>>>> \AccessGrid3\Logs (adjusted for your locale, of course) >>>>> Tom >>>>> >>>>> On Apr 22, 2010, at 11:11 AM, Jesus Cea Oliva wrote: >>>>> >>>>> Hi list again, sorry for my constants problems xD. >>>>> >>>>> I'm trying upload files to our VenueServer and my problem is >>>>> strange. Firstly, I've created a VenueServer locally, for testing. >>>>> >>>>> Well, when I try to upload a file, I can see the message on the >>>>> VenueClient "Uploading file X" and a progressbar, but, when the >>>>> progressbar is near to the end, stops and list of files (Data >>>>> section) don't refresh. So I beleived that the file can't upload. >>>>> >>>>> But, really, the file has uploaded, because I can see the folder >>>>> Data of my local VenueServer, and the file is copied in this >>>>> folder. >>>>> In this case, if I reset VenueServer and VenueClient, then, I can >>>>> see the files in Data section. >>>>> >>>>> So I think that the file uploads fine but, the list of files don't >>>>> refresh and the file uploaded don't appears. >>>>> >>>>> I must say that, sometimes, I can upload a file so I don't know >>>>> whats really the problem (ports closed? firewall? path or name >>>>> files? file size?) . >>>>> >>>>> Thanks again and apologize for any inconveniences >>>>> >>>>> Regards! >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> <VenueClient.log> >>>>> >>>>> Christoph Willing +61 7 3365 8316 >>>>> QCIF Access Grid Manager >>>>> University of Queensland >>>>> >>>>> Christoph Willing +61 7 3365 8316 >>>>> QCIF Access Grid Manager >>>>> University of Queensland >>>>> >>>>> >>>> >>>> Christoph Willing +61 7 3365 8316 >>>> QCIF Access Grid Manager >>>> University of Queensland >>>> >>> >> >> Christoph Willing +61 7 3365 8316 >> QCIF Access Grid Manager >> University of Queensland >> > Christoph Willing +61 7 3365 8316 QCIF Access Grid Manager University of Queensland
