Sam thinks of a number. He doubles it and adds 4 to it. He then multiplies this by his original number and gets $96$. What was his original number?