As you can see from pandoc's changelog, the man reader was only introduced in version 2.4. You're using an older version, so...
- Bug#959851: pandoc: $ in nroff table causes spurious TeX-... James Youngman
- Bug#959851: Acknowledgement (pandoc: $ in nroff tabl... James Youngman
- Bug#959851: Acknowledgement (pandoc: $ in nroff ... James Youngman
- Bug#959851: pandoc: $ in nroff table causes spurious... John MacFarlane
- Bug#959851: pandoc: $ in nroff table causes spur... Jonas Smedegaard
- Bug#959851: pandoc: $ in nroff table causes ... James Youngman
- Bug#959851: pandoc: $ in nroff table cau... Jonas Smedegaard
- Bug#959851: pandoc: $ in nroff tabl... Jonas Smedegaard
- Bug#959851: pandoc: $ in nroff table cau... John MacFarlane

