Overfishing drives population decline, which in turn drives loss of genetic diversity. Many studies provide evidence of declines in genetic diversity; however, controversy exists within the literature, as some studies show evidence of no change in genetic diversity despite decades of overharvesting. The apparent discrepancy in the literature should therefore be examined to understand what biological and ecological processes are driving the differences in results. Here, we assess how different factors contribute to fisheries-induced susceptibility to declines in genetic diversity by first focusing on the different roles of genetic markers. Second, we assess how habitat type and conditions contribute to loss of genetic diversity. Third, we assess how life history and physiology affects catchability and loss of genetic diversity. Finally, we discuss how coinciding abiotic and biotic factors influence the intensity of genetic loss. We find a multitude of these factors could be interacting to influence how results are perceived and how intense the loss of genetic diversity can be. Future studies should carefully consider the methodology of genetic analysis used, as well as considerations of life history and ecology of the target species.