Or he can just tell you that his credit machine is broken and you have to pay by cash.
Charging by credit is a privilege, not a right. There's no law that requires him to let you pay by card.
And of course visa is going to say it's wrong to charge a minimum. They want to make money on every $1 transaction.