The brad point bits or Forstener bits that the others suggested are good ideas.
It is essential that you clamp the work so that it can't move.
If you are having tear-out problems, then support the work from behind. Drill a 7/16 hole in a piece of scrap wood. Slice down the length of the hole with a saw (like a table saw), so that you have a semi-circular support that matches the radius of the wood. If you are drilling through the side of the dowel, then clamp the dowel in the U-shaped channel so that the dowel can't rotate. The dowel now has the back-side support it needs to avoid tearout. If you are drilling the end of the dowel, then sandwich the dowel between the two halves of your scrap piece with the end of the dowel clamped just below the end of the "sandwich". Drill carefully - hopefully the sandwich will prevent a blowout.
If you must use ordinary twist drills, try the smallest drill you have, and work your way up through the sizes until you reach 1/4 inch. You may achieve success by removing small amounts at a time.
Do you want to drill the side of the dowel or the end?
What tool are you using? A drill press? A hand drill?
What kinds of drill bits do you have? Which ones are you using?
Are your drills sharp?