Sand paper, lots and lots of sand paper! Grits ranging from 60 to 800, or up to 1500 for pens if you want a perfect finish. I also use steel wool for finishing.
Safety items such as a face shield.
Tools, be sure to buy what you think you will use/need, stay away from kits, they generally aren't much cheaper and you end up not using half the tools. To start I would invest in a couple of specifics, a roughing gouge, spindle gouge, and skew for pens. A bowl gouge and round nose scraper for bowls. And a parting tool is nice to have for everything.
A pen mandrel and a self centering chuck along with a faceplate or two, and maybe a different live center to use with your pen mandrel or if you are going to do something like pepper grinders you want a nice large cone.
A grinder for sharpening if you don't have one, and a sharpening system which makes getting correct bevels on your tools a lot easier.
A broom and dustpan, you'll hopefully be making lots of chips! And plenty of scrap lumber in the form of 2x4's to practice tool control with.
I want to die quietly in my sleep like my grandfather, not screaming in terror like the passengers of his car.