It sounds like the OP might not fully understand RMODE. RMODE is a directive to the binder and loader of where in storage to place a module. As was mentioned, the granularity is on a section basis, and the directive is applied at module fetch time when store is acquired into which to place the module.
The module is where the module is. If you need to be in AMODE 24 "at some point" then that part of the module must be RMODE 24. The RMODE of a load module or program object is generally the minimum of the RMODEs of the sections that comprise that executable. An exception is, as Ed Jaffe mentioned, when something like RMODE=SPLIT is used on the bind of a program object. In the RMODE=SPLIT case you can end up with one part of the executable in one RMODE and the other part in another RMODE and V-Con's between the two parts are resolved at module fetch time. Peter Relson z/OS Core Technology Design
