> I rarely use LLMs directly, and have never used them for coding, so I > don't know, but: is it not possible to feed the AI policy into the LLM > before you start? It would seem like a flaw if you can't constrain > the kind of output or interaction that you want, or specify the rules > that you need to follow.
Yes, you certainly can and that leads to an interesting idea: why not have the current policy include instructions that can be passed to an LLM to constrain what it can do? > That is, I'd naively assumed that one of the main advantages of having > a written policy was that LLMs would be able to work to it. I was talking to an architect last week who was complaining that when he uses an LLM to write contracts, it often comes up with things that violate his insurance policy. So I suggested that he simply give it the policy and ask it not to violate that. The only problem is that even though the LLM is *capable* of following that policy, there's no guarantee that it actually *will* and this is an issue with LLMs in general. But it will a high percentage of the time and that's probably good enough for this purpose because you'll know when it didn't.
