Accuracy, flexibility, stability and capacity.
Accuracy of the spindle and bearings means the chuck and the bits will run without wobbling and be perpendicular to the table. A quality chuck is the answer usually.
Flexibility, what is the maximum dimension from the center of the chuck to the column? How large a piece can you get under the drill and chuck? Does the table have slots or holes to allow for jugs or fixtures? Sometimes a hole needs to be drilled at an angle. You can either tilt the table, wedge up the work at the desired angle, OR tilt the whole drill head on a rotating column usually called a Radial Arm Drill press. http://toolcenter.com/30-140_Radial_...ill_Press.html
Capacity means how large a shank the chuck accept, usually 1/2".. How much power will the motor and drive system deliver to the chuck without bogging or slipping? And finally, the gripe of most woodworkers, how much travel in the quill .... the more the better. Woodworkers drill deep holes in wood and use longer drills in general than for metal. Having to stop the drill and lower the table to completely bore the hole is a pain.
Floor models will allow taller workpieces under the drill than a bench top model.
Drills are not the only "tool" a drill press will use, there are hole saws and sanding drums. A hole saw takes a lot of power in the larger diameters. The quality of the chuck is important and if it's constantly loosing it's grip, that a pain.
Stability is usually a function of the size of the base and because drill presses are top heavy, the bigger the base, the more stable the machine.
Depth stops are usually a thread rod with an adjusting nut to limit the downward travel. There's not a lot to go wrong with that basic system. The rod is usually flat on both sides to allow the nut to slide up or down without threading. The first clue in looking at a drill press is to notice how long the depth rod is, since that will indicate the quill travel.
I have 2 floor models and 2 bench tops, and use them all depending on the size, type of material and power needed. The range of speeds from the slowest to the fastest, will determine how versatile the machine is for different materials and size drills or accessories. Speed changing via shifting the drive belt or with a variable speed pulley system is vital. The smaller the bit the faster it should spin and vice versa. Metal drilling requires very slow speeds with larger, say 1" diameter, bits than wood drilling of the same size. About 200 RPM in metal and about 1000 RPM in wood, for example.