Re: [lftp-devel] [lftp] Status for incomplete files

2014-10-13 Thread akshay gupta
Thats just great, cheers!! On Fri, Sep 19, 2014 at 5:11 PM, Alexander Lukyanov lavv...@gmail.com wrote: It will be released in one or two weeks. 2014-09-19 15:38 GMT+04:00 akshay gupta akshaygupta...@gmail.com: Thanks a lot Alexander, when will these features come out in release version? I

[lftp-devel] Ways to know number of new files after mirror

2014-10-13 Thread akshay gupta
Hi All, I want to know how many new files are added after mirror command has completed. I am going to run the lftp command through java code using runtime exec() call. Does lftp command provide some special exit status code when mirror succeeds but no new files are added. if not can you suggest

Re: [lftp-devel] possible bug in lftp mirror using xfer:use-temp-file option

2014-10-13 Thread akshay gupta
does that. I'll see if it is possible to reverse the order. 2014-09-30 12:18 GMT+04:00 akshay gupta akshaygupta...@gmail.com: Hi, I am trying to mirror files on Fuse mounted hdfs file system ( https://wiki.apache.org/hadoop/MountableHDFS). Lftp is creating some 0 byte files. Observations

Re: [lftp-devel] possible bug in lftp mirror using xfer:use-temp-file option

2014-10-13 Thread akshay gupta
This option (xfer:use-temp-file) is available in development snapshot http://lftp.yar.ru/ftp/devel/lftp-4.5.5.20140919.tar.gz. On Tue, Sep 30, 2014 at 10:35 PM, akshay gupta akshaygupta...@gmail.com wrote: Its causing unpredictable behaviour in case of fuse mounted file system. Please look

Re: [lftp-devel] possible bug in lftp mirror using xfer:use-temp-file option

2014-10-13 Thread akshay gupta
I am using this feature to know whether a file is completely transferred or lftp is still downloading this file. On Wed, Oct 1, 2014 at 12:46 PM, akshay gupta akshaygupta...@gmail.com wrote: This option (xfer:use-temp-file) is available in development snapshot http://lftp.yar.ru/ftp/devel/lftp

[lftp-devel] providing key file for ssh in fish protocol

2014-10-14 Thread akshay gupta
Hi All, I want provide a key file for authentication in fish protocol. Fish internally uses ssh for connection and ssh have a support for providing key files for password less authentication, how can I provide key file using lftp for FISH protocol. Thanks, Akshay

[lftp-devel] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
Hi, I am trying to mirror files using lftp, I have tested a use case where a file at the source machine gets modified but the file name remains the same. After mirroring, lftp is unable to replace the old file and also leaves a temporary file. For example. If source have file a.txt and its

Re: [lftp-devel] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
. --- 221 Thank you for using the FTP service on 192.168.128.124. Closing control socket ~Akshay On Mon, Oct 27, 2014 at 1:18 PM, Alexander Lukyanov lavv...@gmail.com wrote: Can you provide a debug log? Why lftp cannot replace the old file? 2014-10-27 10:44 GMT+03:00 akshay gupta akshaygupta

Re: [lftp-devel] [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
no, I am running as a root user. There is some issue with lftp, when file with same name already exist its not able to rename the temp file to actual file. On Mon, Oct 27, 2014 at 1:44 PM, Alexander V. Lukyanov l...@netis.ru wrote: On Mon, Oct 27, 2014 at 01:30:42PM +0530, akshay gupta wrote

Re: [lftp-devel] [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
...@gmail.com: I have found the bug. As a workaround, use set xfer:clobber yes. 2014-10-27 13:06 GMT+03:00 akshay gupta akshaygupta...@gmail.com: no, I am running as a root user. There is some issue with lftp, when file with same name already exist its not able to rename the temp file

Re: [lftp-devel] [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-27 Thread akshay gupta
. Lukyanov l...@netis.ru wrote: On Mon, Oct 27, 2014 at 05:53:19PM +0530, akshay gupta wrote: now its producing 0 byte files and i am also getting input/output error here are the debug logs: copy: renaming `/mount/user/tom/test.txt_tmp_' to `test.txt' mirror: rename(/mount/user/tom

Re: [lftp-devel] [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-28 Thread akshay gupta
at 11:50 PM, akshay gupta akshaygupta...@gmail.com wrote: I am using fuse mounted hadoop distributed file system(HDFS). Can you suggest what changes need to be done to make it work as this is a very critical requirement for my project. Will deleting file and then renaming temp file help? how

Re: [lftp-devel] [lftp] Bug in lftp mirror when using xfer:use-temp-file option

2014-10-28 Thread akshay gupta
/a/cloudera.org/forum/#!msg/cdh-user/R1GR2Asj_lc/f_uuIjG5R1UJ Please try this workaround. 2014-10-28 10:29 GMT+03:00 akshay gupta akshaygupta...@gmail.com: will giving --delete-first option in lftp mirror help in my case? I think its not working in case of use-temp-file option. As per the man page

[lftp-devel] Possible Bug in lftp mirror using mirror:sort-by option

2014-10-29 Thread akshay gupta
Hi, I am trying to mirror files using mirror:sort-by option set to 'date-desc'. My file structure is like this $ ls -l -R total 3191264 drwxr-xr-x 3 vikasgoyal biadmin 102 Oct 22 12:35 1 -rw-r--r-- 1 vikasgoyal biadmin 0 Aug 1 01:00 tmp.txt -rw-r--r-- 1 vikasgoyal

Re: [lftp-devel] Possible Bug in lftp mirror using mirror:sort-by option

2014-10-29 Thread akshay gupta
...@gmail.com wrote: Mirror in lftp works directory by directory. It does not build full list of files first, so it cannot sort the whole list. Maybe the mirror algorithm should be redesigned to build the whole recursive list first. What do you think? 2014-10-29 10:07 GMT+03:00 akshay gupta

Re: [lftp-devel] Possible Bug in lftp mirror using mirror:sort-by option

2014-10-30 Thread akshay gupta
lftp have options like --depth-first and --no-recursion I think it will make sense to have an option like --iterative for iterative file transfer. This option will fetch the full file list and then fetch them iteratively. On Wed, Oct 29, 2014 at 2:29 PM, akshay gupta akshaygupta...@gmail.com

Re: [lftp-devel] Possible Bug in lftp mirror using mirror:sort-by option

2014-11-17 Thread akshay gupta
lavv...@gmail.com wrote: I think it is possible and the amount of programming work is moderate. The recursive model of mirror will stay but the file transfers will be deferred and done on the top level when recursive directory traversal is finished. 2014-11-12 8:32 GMT+03:00 akshay gupta

[lftp-devel] Compressing data at destination machine

2014-11-25 Thread akshay gupta
Hi, I want to store the data that is being pulled by lftp mirror job in a compressed format. Is it possible that mirror function can check file download status from inside of that tar, zip etc. Is it possible to implement such a functionality in lftp? Thanks, Akshay

Re: [lftp-devel] [lftp] Compressing data at destination machine

2014-11-27 Thread akshay gupta
, akshay gupta wrote: I want to store the data that is being pulled by lftp mirror job in a compressed format. Is it possible that mirror function can check file download status from inside of that tar, zip etc. Is it possible to implement such a functionality in lftp? I don't think mirror

Re: [lftp-devel] [lftp] Compressing data at destination machine

2014-12-02 Thread akshay gupta
support writes. Thanks, Akshay On Fri, Nov 28, 2014 at 1:06 PM, Alexander V. Lukyanov l...@netis.ru wrote: On Fri, Nov 28, 2014 at 11:55:07AM +0530, akshay gupta wrote: I dont want lftp to extract data from archives, I want it to compress it into archive so that it occupies less space

Re: [lftp-devel] Possible Bug in lftp mirror using mirror:sort-by option

2014-12-02 Thread akshay gupta
Are these feature present in LFTP implementation timeline? By when will they be implemented? ___ lftp-devel mailing list lftp-devel@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp-devel

Re: [lftp-devel] Possible Bug in lftp mirror using mirror:sort-by option

2014-12-05 Thread akshay gupta
don't want to build from git: http://lftp.yar.ru/ftp/devel/lftp-4.6.0.20141205.tar.gz 2014-12-05 9:43 GMT+03:00 akshay gupta akshaygupta...@gmail.com: Its not working: I run the ./autogen.sh --without-gnutls --with-openssl command and it cannot seem to find Makefile.in in the configure step

[lftp-devel] Handling for small files

2015-02-24 Thread akshay gupta
Hi, I am trying to transfer about 80GB of data using mirror option. File sizes are very small about 0.5 KB to 3 KB per file. Transfer is taking a lot of time which I am attributing to small file size. Is there any option in lftp which I can use to increase the transfer speed. May be some sort of

Re: [lftp-devel] Handling for small files

2015-02-24 Thread akshay gupta
I am using fish protocol. On Tue, Feb 24, 2015, 4:23 PM Alexander Lukyanov lavv...@gmail.com wrote: Which protocol do you use? вт, 24 февр. 2015, 13:48, akshay gupta akshaygupta...@gmail.com: Hi, I am trying to transfer about 80GB of data using mirror option. File sizes are very small

Re: [lftp-devel] Possible Bug in lftp mirror using mirror:sort-by option

2015-02-24 Thread akshay gupta
. You'll have to get gnulib too. четверг, 4 декабря 2014 г. пользователь akshay gupta написал: I have cloned the lftp repo, can you please tell me the steps to create configure file and make script. I tried automake --force-missing --add-missing but its giving src/Makefile.am:45: NEED_TRIO