How do i Focusing on the breath and letting thought go

Now this may sound silly but I'm new to mediating and I hear that when you have a thought you just let it go. But if I'm focusing on the breath do I go off it, accept that I'm thinking then go back to the breath or just stay on the breath and if I get distracted go right back to it and not even focus on my thoughts?