Hi,I did a very basic comparison of download speed. I used similar "curl .." command to download a large file (13.6 GB) and gathered the numbers. Looks like WebHDFS with Knox is very slow ( at least 20x slower). I ran it twice with similar numbers. For Knox, I turned off SSL and both cases I used unsecured (non-Kerberos) cluster. Download with Knox took nearly 49 minutes whereas direct download took 2 mins. The download speed was 4811k for Knox and 99.6M for direct download. I'm sure I have done something wrong. Do you see any such performance? Any help will be really appreciated. Regards,Mohammad
Interactions:curl -o t2.direct -L http://<WEBHDFS_HOST>:50070/webhdfs/v1/<FILE_PATH>?op=OPEN % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0100 13.5G 100 13.5G 0 0 99.6M 0 0:02:19 0:02:19 --:--:-- 117M curl -H X-Auth-Params-Email: [email protected] -o t2 -L http://<KNOW_HOST>:8445/gateway/sandbox/webhdfs/v1/<FILE_PATH>?op=OPEN % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 13.5G 0 0 4811k 0 --:--:-- 0:49:12 --:--:-- 6121k
