Both of the previous suggestions are great. Getting a perfect fit on uneven walls is always a little frustrating. Since you have already built the cabinet the above solutions fairly easy options unless you are open to using scribe mould. Personally I don't mind scribe at all, often it adds an extra dimension to the side of the built-in. However, when I do one that I don't want to or can't use scribe I will measure meticulously and cut my outer stiles first to get the proper profile(often beveling the backside to cut into the drywall just slightly). Then I will take my measurements off of the plumbed and profiled stiles and build from there (but it is ALWAYS a pain to get it into place when built, also why I usually put the face frame on after the boxes are installed). Anyway, another alternative is to rip filler strips(at a taper if the gap is not even) and apply a little glue then use a small block like the end of a shim, and tap the strips into the side until they are snug. Then take a razer knife and cut whatever is still sticking out (I like to rip them at a slight bevel with the larger end just wider then the gap so they get a snug fit). Then just sand it and apply your finish. But in your case I would just rip (or buy if you prefer) some scribe and use it to cover any gaps, its the easiest fix. Remember to be extra careful when ripping strips, more people lose thumb tips that way.