One thing I did with some younger kids is get a soft spongey like ball have them set up in the batters box with a bat and throw the ball at them. Once they get confident that they can get out of the way of most pitches they become more confident in the batters box.
Getting over that fear takes time and patience from the coach or parent so be undestanding.