I'm not sure if this changed with IRA, but at least with the old reload code, if the instruction doesn't match, reload will look for the alternative with the cheapest total cost of alternative and reloads. Unfortunately, the fact that some reloads might be impossible does not figure in the decision. Where the costs are equal, the order is the tie-breaker; the first alternative is choosen then. You must make sure that the HI regs alternative gets precedence over the HL regs one when the operand getting the HI or HL regs is reloaded. Uou can do this by tweaking alternative costs, alternative order, REG_MOVE_COST (for the reload costs), or any combination if these.
Or better, fix the alternative choosing algorithm to avoid alternatives which are already known to be impossible to fulfill when looking at the registers that are live during the insn and the constraints of a single operand in isolation. I.e. if we require n hard registers of a particular class, check that there are n hard registers free in that class. Probably discounting the possibility of having different overlapping classes between primary output and secondary / tertiary reloads which can fit individually but not together. (I.e. leave it to the port writer to avoid that). To avoid slowing the compiler down too much, you might want to restrict this to CLASS_LIKELY_SPILLED and/or delay the test till the alternative is choosen as the best one, and if the test fails, re-do the alternative choosing with a bit mask of discarded alternatives. (In which case a combined register/memory alternative that uses impossible address reloads on the first round will be discarded so the register part is also disabled - but I suppose that's a limitation we can live with.) It may be noted that there is nothing really stopping us from considering all operands of an alternative - for each operand, reload decides on an actual register class requirement, even if the alternative has multiple subparts - but I think it would make the code much more complex, and I doubt it would buy much in making ports easier to write or the generated code better.
