Is austerity necessary in Buddhism?

I have been watching Buddhist followers for sometime now in India and it made me wonder if austerity is necessary in Buddhism ? Like if I wish to become a Buddhist do I need to give up on listening to music, watching tv, using social networking sites, making a livelihood by working in a firm, self- pleasure (read masturbation), have to sleep on floor and beg for a living? I don't mean any of this questions in an offense way. But most of the people here, at least the common opinion as most of my friends/family have told me is that if you believe in Buddhism you have to live like a "Monk".