Let me start by saying I haven't done box joints before, but I've been reading about them for a while. They seem easy and I'm sure I have a grasp on them. That being said i'm just trying to help you brainstorm this issue. So if I sounds like am idiot i'm sorry lol.
With your pictures it looks like all if you're cuts are exactly the same, but your fingers are growing on you. The only cause for that is that your pin is increasing distance from your blade. Several things can cause it. to me it looks like the culprit is your clamps holding the fence with the pin. they probably work fine, but as you push the piece against your pin, your fence can slightly move. I've had the same problem with my router fence, I used to clamp it onto my melamine top and it would shift. I fixed it by taping sandpaper in between the fence and top. It seems to have fixed it, plus I don't push so hard against the fence. I suggest resetting your fence, then screwing it together, you can use slots and wing nuts so you can adjust it, or just screw it. Double check the measurements, and recut your pieces.
You can save your boards too. either make small pieces and glue them into your dados, or just recut as it is (being extremely careful since it won't sit tight on your pin this way), fit it together, then fill in the gaps. I would do it the first way though. the fix won't be too noticeable. It doesn't look like all of your boards/cuts are bad though, so go through carefully measuring them all. You may only need to fill in half of your cuts.
Woodworking isn't about being perfect, the joy is learning from mistakes and finding ways to fix them. I do it all the time. A few minutes ago I realized I miss measured a mortise. I lengthened it then will fill in the gap after its glued together.