dos, if you finish it like the rest of the sink there should be no problems with fuzziness or splintering. If the drain flange wiggles around and rubs a lot, there might be an issue, but that shouldn't happen. The drain, if installed properly, will be stable and not shift in its position so the hole will be sufficiently protected from rubbing that would cause splintering or fuzziness. I'm not sure how you'd get dirt there, but again, that should be a non-issue if the drain is installed properly.
I'll recommend using new router bits for the hole though as that will help reduce any initial splintering. If you do get some splintering, glue things down then do your finish work on the hole and you should be fine.