According to a recent study by Nerdwallet, here, only about half of high school seniors completed the FAFSA in 2013. Admittedly not all high school seniors go on to college, but much of the aid that is awarded through the FAFSA, including Pell Grants, can be used to pay community college and vocational school costs.
In addition, you need to complete the FAFSA in order to be eligible for federal loan programs including Stafford and PLUS loans.
Nerdwallet estimates that failure to complete the FAFSA resulted in over $2.9 billion in free (i.e. grant) money for college being left on the table. So although you may have plenty of company if you choose not to complete the FAFSA, it would not be good company to be in.