I can think on two options.
*1. Unique Key*
db.define_table("table",
Field("table_a", "reference table_a"),
Field("table_b", "reference table_b"),
Field("unikey", unique=True, notnull=True, compute=lambda row:
"%(table_a)s-%(table_b)s" % row)
)
*2. Form validator*
def check_unique(form):
if db((db.table.table_a == form.vars.table_a) & (db.table.table_b ==
form.vars.table_b)).count():
form.errors.table_a = "You cannot insert or edit a duplicate
combination"
form = SQLFORM(db.table).process(onvalidation=check_unique)
Mybe it can be implemented as a Field Validator, have to try.
--