Hi Mohammad - Thanks for reporting this.
That is a big difference. Let me play around with it and see what I can reproduce. thanks, --larry On Sat, Nov 5, 2016 at 5:52 PM, Mohammad Islam <[email protected]> wrote: > Hi, > I did a very basic comparison of download speed. I used similar "curl .." > command to download a large file (13.6 GB) and gathered the numbers. > > Looks like WebHDFS with Knox is very slow ( at least 20x slower). I ran it > twice with similar numbers. For Knox, I turned off SSL and both cases I > used unsecured (non-Kerberos) cluster. > > Download with Knox took nearly 49 minutes whereas direct download took 2 > mins. The download speed was *4811k* for Knox and *99.6M* for direct > download. > > I'm sure I have done something wrong. Do you see any such performance? Any > help will be really appreciated. > > Regards, > Mohammad > > > > > > > Interactions: > curl -o t2.direct -L http://<WEBHDFS_HOST>:50070/ > webhdfs/v1/<FILE_PATH>?op=OPEN > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > 0 > 100 13.5G 100 13.5G 0 0 *99.6M* 0 0:02:19 *0:02:19* > --:--:-- 117M > > > > > curl -H X-Auth-Params-Email: [email protected] -o t2 -L http:// > <http://hadoopdevgw01-/><KNOW_HOST>:8445/gateway/sandbox/webhdfs/v1/< > FILE_PATH>?op=OPEN > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > 0 > 0 0 0 13.5G 0 0 *4811k* 0 --:--:-- *0:49:12* > --:--:-- 6121k > >
