The relative age effect (RAE), which refers to an over-representation of selected athletes born early in the selection year, was proven to be present in alpine ski racing in all age categories at both national and international levels. However, the influential factors on, or the causal mechanisms of, the RAE are still unknown. Therefore, the aim of the present study was to examine three possible influential factors on the relative age effect in alpine skiing: physical performance, anthropometric characteristics and biological maturational status. The study included the investigation of 282 elite Austrian youth ski racers and 413 non-athletes (comparison group) of the same age (10–13 years) and region. Six physical performance tests were performed, body mass and height were assessed, and the age at peak height velocity (APHV) was calculated. A significant RAE was present in the ski racers. No differences were shown in the physical performance characteristics or in the calculated APHV between the relative age quarters. These results suggest that ski racers born in the last quarter can counteract the relative age disadvantages if they already present the same level of physical performance and maturational status as those born at the beginning of the year. The height and weight of ski racers born at the beginning of the year were significantly higher compared to the non-athletes, and ski racers born in relative age quarter 1 were taller and heavier compared to the ski racers of the other quarters. This indicates that the anthropometric characteristics influence the selection process in alpine ski racing, and that relatively older athletes are more likely to be selected if they exhibit advanced anthropometric characteristics.