That depends - what resource did it consume, concerning your admins? CPU? Were you uploading using the chunked technique?
On Fri, Sep 4, 2015 at 9:06 PM Shashi Vishwakarma <[email protected]> wrote: > Hi > > I have been using curl command for ingesting data into HDFS using WebHDFS. > Admins are saying that resource utilization for curl is very high which is > hampering server performance. > > I would like to know if curl can use so much of resource which could lead > to performance issue. > > Just to give more idea , I have been ingesting around 25Gb of data into > HDFS. Not to sure if curl can cause so much of issue with such a small data > set. > > Any suggestion would be a great help. > > Thanks, > > Shashi >
