The decline in adolescents’ physical fitness (PF) in recent decades has raised concerns about current population’s possible future challenges with health and physical functional capacity. This study explored the associations between body composition, physical activity, maturation, and PF development in adolescents. Furthermore, PF development of adolescents with low initial PF was assessed. A 2‐year observational study was conducted between spring 2013 and 2015. Nine comprehensive schools and their 10‐ to 13‐year‐old students were invited to participate in the study (1778), and a total of 971 students (54.6%) agreed. Cardiorespiratory fitness (20‐metre shuttle run), muscular fitness (push‐ups), fundamental movement skills (5‐leaps test), body composition (bioelectrical impedance analysis), moderate‐to‐vigorous physical activity (accelerometer) and pubertal status (self‐assessment questionnaire) were measured at 1‐year intervals. Latent growth curve modeling (LGM) was used to study PF development over time. Change in fat mass had the strongest and most coherent associations with PF development during adolescence. Fat‐free mass, moderate‐to‐vigorous physical activity and pubertal status were associated with PF development, although not systematically. Subgroup analyses showed that PF development in the low fitness group followed a similar pattern as the whole population. However, their PF remained significantly lower throughout the 2‐year period. The findings suggest that fat accumulation is an essential detrimental factor for PF development during adolescence. Actions to prevent excessive fat accumulation might help to prevent future declines in functional capacity. Indications that low fitness levels sustain during adolescence highlight the relevance of detecting these individuals and providing interventions already before adolescence.