New Optical and Near-Infrared Surface Brightness Fluctuation Models. II. Young and Intermediate-Age Stellar Populations