Here are my concerns: There is a lot of 3D detail, like the pressed back design and fluted parts of the chair. There doesn't seem to be much in terms of varnish or shellac, but the wood is stained. I'm concerned that if I try and strip it down to bare wood, I'll have lots of areas where I just can't get there due to the little nooks and crannies. I was going to just try and clean it as best I can, then stain the bare areas to match the rest.
What do you think of that approach?
Your initial thoughts of just cleaning off oils, and light debris would work with just using mineral spirits. It won't remove any finish, and would leave the surface in a better state to determine if some "touch up" staining can be done, instead of a complete stripping. Once clean, you could try stains or dyes for "fill ins", or if more color is needed, gel stain would work well.
If you decide you need to strip off the finish, start with the least toxic, like CitraStrip. It's a waterbase stripper that can be used indoors, and has an easy clean up. If that doesn't cut it (literally), try lacquer thinner as a wipe/rub. That should remove most film finishes. If that doesn't cut it, go to an MC (methylene chloride) based stripper. It's very toxic, and you would need to closely follow the directions.
I wouldn't use a power washer, as it has to be used outside. It will displace toxic chemical all over the place...on the grass, concrete, or asphalt. It can kill foliage. It will spritz chemical on yourself, and it can burn through the skin. If it gets into your eyes you could be blinded.
It can dislodge previous repair work, loosen joints and dislodge veneer if present. It will pressure force water into the pores of the wood, changing the moisture content. That could cause expansion of the wood and joints, and when the wood finally dries out, could crack. It also will raise the grain.