O.K. I have to say it, fly cutters are intended for drill presses. Every professional machinist I've asked about them say they are the most dangerous tool they know of.
I can't imagine holding a hand drill stable enough to get a round hole with one. A good hole saw is the tool for hand work. Tom Fowle