Jason, if you just create a new type representing integers mod N, with the appropriate arithemetic operations defined for it, then if you make a matrix of these numbers the built-in matrix operations work. In fact, this has already been done; see:
https://github.com/andrewcooke/IntModN.jl
