It would be better to move the fence away from you so the bit wouldn't be climbing and move the wood from right to left. Actually if the part is wide enough it would be better to put a fence on both sides of the board so it's not trying to pull away from the fence.