In order to display something in IJulia, you don't have much choice but to redraw the whole figure I think, since it needs to render a new PNG image and send it to the front-end.
On the other hand, if you are displaying in a separate Matplotlib GUI window (via pygui(true)), then there are various matplotlib functions to change some but not all of the data, which might be faster. But this is more of a matplotlib question than a Julia/@manipulate question.
