Hi, I am working on building some responsive UI to visualize reports on top of Zeppelin. I found that running the same query using %spark sometimes is significantly slower than using %sql.
For the query I am running, I can get result within 1 sec using %sql, but sometimes using more than 2 secs to get result using %spark. %spark is also very responsive after a clean restart, but become slower after it runs for a while, but %sql is still fast. Does anyone have the same issue? What is the difference between running using %spark and running using %sql? Thanks! Zhong
