Habitat loss and fragmentation are major threats to biodiversity and ecosystem processes. Our current understanding of the impacts of habitat loss and fragmentation is based largely on studies that focus on either short-term or long-term responses. Short-term responses are often used to predict long-term responses and make management decisions. The lack of studies comparing short-and long-term responses to fragmentation means we do not adequately understand when and how well short-term responses can be extrapolated to predict long-term responses, and when or why they cannot. To address this gap, we used data from one of the world's longest-running fragmentation experiments, The Wog Wog Habitat Fragmentation Experiment. Using data for carabid beetles, we found that responses in the long term (more than 22 yr post-fragmentation 22 generations) often contrasted markedly with those in the short term (5 yr post-fragmentation). The total abundance of all carabids, species richness and the occurrence of six species declined in the short term in the fragments but increased over the long term. The occurrence of three species declined initially and continued to decline, whilst another species was positively affected initially but decreased in the long term. Species' responses to the matrix that surrounds the fragments strongly predicted both the direction (increase/decline in occurrence) and magnitude of their responses to fragmentation. Additionally, species' responses to the matrix were somewhat predicted by their preferences for different types of native habitat (open vs. shaded). Our study highlights the degree of the matrix's influence in fragmented landscapes, and how this influence can change over time. We urge caution in using short-term responses to forecast long-term responses in cases where the matrix (1) impacts species' responses to fragmentation (by isolating them, creating new habitat or altering fragment habitat) and (2) is likely to change through time.