a few things
Are you using a splitter or riving knife? If so, that will help retain the board against the fence.
If the boards are narrow, under 6", it's more difficult to apply pressure in towards the fence. You should place your feed hand or push stick as close to the blade as you feel safe and that will give you some added "leverage" when pushing in and forward. I hook my thumb over the edge of the board to give added leverage. Boards under 6" I use a push shoe.
You must start with a straight edged board! :yes:
A curved edge board will not register against the fence, it will just follow the curve that already there.
A straight line jig is the fastest way I know to get a get straight edge, faster than a jointer, especially on longer boards.