The next time you watch a game note how much time gets wasted between innings. I see team where the catcher makes the last out and they have 3 subs and 4 coaches and no one goes to warm up the pitcher. Teams where the next hitter takes forever to get up to bat. 10 seconds here, 30 there, adds up to a few minutes each game. When you are down a run and wish you had another 3 minutes you know where it went.
Which is why we go 1 and down with no infield balls when we take the field. If the pitcher is not loose after the first pitch then the next 4 are probably not going to help.