Besides cognitive factors, children's learning at school may be influenced by more dynamic phenomena, such as motivation and achievement-related task-avoidant behavior. The present study examined the developmental dynamics of task-avoidant behavior and math performance from kindergarten to Grade 4. A total of 225 children were tested for their arithmetic skills in kindergarten and in Grades 1, 2, and 4 of elementary school. Children's task-avoidant behavior in learning situations was rated by their teachers. The results of latent growth curve analyses showed that math performance and task-avoidant behavior develop in tandem: an increase in task-avoidant behavior was related to less improvement in math performance. Furthermore, a high initial level of task-avoidant behavior predicted less improvement and slower improvement in math later on.