The aim was to study objectively assessed walkability of the environment and participant perceived environmental facilitators for outdoor mobility as predictors of physical activity in older adults with and without physical limitations. 75–90-year-old adults living independently in Central Finland were interviewed (n = 839) and reassessed for self-reported physical activity one or two years later (n = 787). Lower-extremity physical limitations were defined as Short Physical Performance Battery score ≤9. Number of perceived environmental facilitators was calculated from a 16-item checklist. Walkability index (land use mix, street connectivity, population density) of the home environment was calculated from geographic information and categorized into tertiles. Accelerometer-based step counts were registered for one week (n = 174). Better walkability was associated with higher numbers of perceived environmental facilitators (p < 0.001) and higher physical activity (self-reported p = 0.021, step count p = 0.010). Especially among those with physical limitations, reporting more environmental facilitators was associated with higher odds for reporting at least moderate physical activity (p < 0.001), but not step counts. Perceived environmental facilitators only predicted self-reported physical activity at follow-up. To conclude, high walkability of the living environment provides opportunities for physical activity in old age, but among those with physical limitations especially, awareness of environmental facilitators may be needed to promote physical activity.