The numbers only represent the fan speed. Even when set to "0", or fan off, the ventilation system is still physically open.
That's why the faster you go, the more air comes out. The intake is in a relatively high-pressure area (same place they put cowl-induction intakes), so increased vehicle speed generates increased air movement through the vents.
To stop the air completely, you'll have to physically close the system - shut the dash vents.