I have used dowels many times. They are for me an easy method for either aligning long boards, or reinforcing the joint for end grain to face/side grain.
For edge work, my jig sits on top of the edge of the piece and makes contact with the wood. I do not get tear out. I normally use twist drill bits, mostly due to my stop collars being able to screw to the bit better than my dual spiral brad point bits.
If the stop is set correctly, I get a consistent hole. I make my holes a little more than 1/2 the length of the dowel.
I always do a dry fit. I want to ensure the dowels will fit snug without the gap you observed, and if to ensure the dowel is not too snug which can impact the glue flow and strength of the joint.
I normally do a light hand sanding on my dowels.
For drilling on the face/side grain where my edge jig will not work, I drill the edge holes first, then insert dowel centers to mark the hole locations on the face/side grain board.
I drill the face/side grain holes on the drill press. I normally use the same twist drill, sometimes I may use my brad point drills.
I rarely use pine, but it is not uncommon to have some tearout. Sometimes slower works better than faster, sometimes faster works better than slower. Not easy to figure. Just test on scrap first.
Good luck with you next use of dowels. I would not let the first attempt put you off.