You have to be ready to call his bluff. Tell him that you do not go out with beardy-boys! If he insists, tell him he has a choice to make--you or the beard. Tell him that if you wanted to date Osama bin Laden, you would go to Afghanistan!
Now, when you do this, you have to be ready to follow through if he chooses his beard over you.