What you say is correct.
The argument against software patents used by the FSF isn't actually that
they restrict user freedom, but that they create a web of vaguely defined
'no-go' zones which make it nearly impossible to implement anything without
being at risk of a lawsuit. Patented hardware designs aren't 'free', but the
risk of an innocent developer accidentally running afoul is much lower than
for a patented software.
That said, allowing people to learn from a creation whilst locking out
competitors is difficult. Blocking commercial development entirely may not be
a good idea, as jxself pointed out; if patenting is necessary, the best idea
might be demanding a fixed portion of any profits made in return for the
right to use the design. However, this still has its flaws.