You said it yourself. Even if your 256MB machine were typical (it's not), the less cache memory you use to cache dentries of /usr/lib, the better (more memory for your apps, or to cache other, more useful stuff).
If you suspect that separating /usr/lib into multiple directories will improve performance, please bring reasonable benchmarks, not speculation.
