Nemesis is a hypothetical red dwarf star or brown dwarf, orbiting the Sun at a distance of about 50,000 to 100,000 AU, somewhat beyond the Oort cloud. This star was originally postulated to exist as part of a hypothesis to explain a perceived cycle of mass extinctions in the geological record.
In 1984 paleontologists David Raup and Jack Sepkoski published a paper claiming that they had identified a statistical periodicity in extinction rates over the last 250 million years using various forms of time series analysis. They focused on the extinction intensity of fossil families of marine vertebrates, invertebrates, and protozoans, identifying 12 extinction events over the time period in question. The average time interval between extinction events was determined as 26 million years. At the time, two of the identified extinction events (Cretaceous-Tertiary and Late Eocene) could be shown to coincide with large impact events. Although Raup and Sepkoski could not identify the cause of their supposed periodicity, they suggested that there might be a non-terrestrial connection. The challenge to propose a mechanism was quickly addressed by several teams of astronomers.
Two teams of astronomers, Whitmire and Jackson, and Davis, Hut, and Muller, independently published similar hypotheses to explain Raup and Sepkoski's extinction periodicity in the same issue of the journal Nature. This hypothesis proposes that the sun may have an as yet undetected companion star in a highly elliptical orbit that periodically disturbs comets in the Oort cloud, causing a large increase in the number of comets visiting the inner solar system with a consequential increase in impact events on Earth. This became known as the Nemesis (or, more colorfully, Death Star) hypothesis.
If it does exist, the exact nature of Nemesis is uncertain. Richard A. Muller suggests that the most likely object is a red dwarf with magnitude between 7 and 12, while Daniel P. Whitmire and Albert A. Jackson argue for a brown dwarf. If a red dwarf, it would undoubtedly already exist in star catalogs, but its true nature would only be detectable by measuring its parallax; due to orbiting the Sun it would have a very low proper motion and would escape detection by proper motion surveys that have found stars like the 9th magnitude Barnard's star.
The last major extinction event was about 5 million years ago, so Muller posits that Nemesis is likely 1-1.5 light years away at present, and even has ideas of what area of the sky it might be in (supported by Yarris, 1987), near Hydra, based on a hypothetical orbit derived from original apogees of a number of atypical long-period comets that describe an orbital arc meeting the specifications of Muller's hypothesis.
The idea of periodic extinction rates is pretty neat, at least from No Chance It'll Happen Tomorrow point of view (although, maybe 2012, if we're all real lucky), and the whole idea of a giant, unproven, hidden star, only makes it slightly better. It's a kind of mad science, the type of thing that results in bad, bad Hollywood films.