I’ve recently gone through my own unhappy epoxy results and can offer some general rules of thumb:
• ALWAYS check with the manufacturer RE compatibilities and these sorts of unexpected results.
• Scuff sanding between coats is intended to provide mechanical “keying” for the following coat, so the “key ways” (scratches) need to be substantial, and 120 - 150 grit is the right grade for that. Scuffing is NOT removing the coating, it is intended to simply roughen and remove the gloss from the surface — go over it quickly and lightly by hand.
• As already noted, epoxy requires UV protection, and a few coats of marine grade spar varnish are the best choice. The varnish is a sacrificial coating — it will have to be sanded back a bit and reapplied every few years as it ages and gets cracked, cloudy, or peels.
On an item like your birdhouse that doesn’t get too physically abused, you might want to save yourself the expense of the epoxy and just go with the spar varnish.