You can certainly wear shirts under the jersey. The rulebook says that anything exposed must "be solid color" and "be the same color". I assume this means that all players wearing an exposed undershirt must all wear the same color. However, in 10+ years doing this, I've never seen this enforced. Further, anyone making a fuss about this in cold weather would come off as a first-rate jackass. Players also put on random hoodies when it's cold. As long as they declare their number when batting, I've never seen any umpire or opposing coach ever push back.