I usually make sure they are finished with their warm up just 5-10 minutes before game start. I have no real reasons except it just seems to make sense they are warmed up and ready right before they have to pitch. Also each pitcher takes a different amount of time to warm up so you need to take that into account as well.