Olfactory decline represents one of the earliest signs of Alzheimer’s disease (AD), and deficits in olfactory identification have now been identified in persons with amnestic mild cognitive impairment (aMCI). Whether the olfactory identification deficit in AD reflects underlying degraded semantic knowledge or lower-order olfactory deficits is uncertain. To address this question, we focused on the kinds of errors committed when participants were given a unirhinal olfactory identification task at baseline and after 1 year. The aim was to assess whether more errors were committed when the target smell is either semantically related or unrelated to the distracters. Fourteen AD, 13 aMCI, and 10 control participants were tested using a modified version of the University of Pennsylvania Smell Identification Test. Examination of error types showed that the control group predominantly selected distracters which were related to the target; in contrast, distracters that were unrelated to the target odor were selected as frequently as related odors by the AD and aMCI groups, for both nostrils. This pattern was maintained 1 year later, and previously designated aMCI patients who then met criteria for AD were more inclined to choose the unrelated distracters than were aMCI patients who did not meet AD criteria. Olfactory identification deficits in AD and aMCI plausibly reflect deterioration in ability to access olfactory-mediated semantic knowledge. This pattern of errors may help distinguish the olfactory identification deficits observed in AD from olfactory identification deficits observed in other conditions not associated with semantic loss.