I reaaly hate to kill the little critters, but it's them or me.
I've had limited success with WD-40, but once again here is my annual post
about my surefire wood boring bee killing routine:
Get a spray can of "Goof Off". Identify live holes during the day. At dusk,
spray Goof Off in the holes. If the bees even make it out of the holes,
they'll fall out dead.
But if other bees are still in there and survive it, or if Larva live
through it and mature inside that hole, they'll push the dead ones out, so
keep watching. It ain't over 'til it's over.
Daddybob