I just started about a month ago, so I have recently been in your shoes. Here is what I've put together in my short time.
Beyond the lathe, which I assume you already have, I would go with a starter pen kit like they have at Rockler here:
It's roughly $65, but when I started adding everything up in my basket separately, this option made the most sense. Depending on the lathe, be sure to get the right pen mandrel. The one that comes with the set I linked is an MT2, and I needed an MT1.
You need the tools, mostly the roughing gouge. I bought this set, which is on sale at Rockler (I don't work for them, I simply go there a lot):
These tools have been all I've needed for my turning so far. It's a good starter set without a big initial cash outlay.
Get a set of sanding pads like these, which is the key to getting your pens as smooth as possible:
If you don't want to spend the money on a pen press, you can use a mini-clamp to press the parts together. You also don't have to spend money on a pen blank drilling vise. You can make one yourself out of two pieces of 2X4 and a door hinge, and it works just as good. If you would like, I can tell you how to make one that is just as effective as a store-bought tool.
You want to get both thin and medium CA glue and some accelerator to put a good finish on the pens, and you want to get some friction polish. I use the HUT Crystal Coat polish as a final finish. It runs about $10 for a 6 oz. bottle.
You also want to buy a good face shield for turning. You might not look cool in it, but it protects you better than just glasses.
Remember that different pen sets require different bushings and different drill bits, so that can be added pieces, too. But I think that's all I have for right now. I'm not saying I have all the answers on this, and there could be better options for things. But I just started, too, and that is what I've bought over the past month, and the results have been solid so far. Have fun!