Philosophers and scientists at Britain's Cambridge University think the question deserves serious study. A proposed Center for the Study of Existential Risk will bring together experts to consider the ways in which super intelligent technology, including artificial intelligence, could "threaten our own existence," the institution said Sunday.
"In the case of artificial intelligence, it seems a reasonable prediction that some time in this or the next century intelligence will escape from the constraints of biology," Cambridge philosophy professor Huw Price said.
When that happens, "we're no longer the smartest things around," he said, and will risk being at the mercy of "machines that are not malicious, but machines whose interests don't include us."
Fears that machines could overtake humans have long been the subject of science fiction - the computer HAL in the movie "2001: A Space Odyssey," for example, is one of film's best-known computer threats.
Price acknowledged that many people believe his concerns are far-fetched, but insisted the potential risks are too serious to brush away.
"It tends to be regarded as a flakey concern, but given that we don't know how serious the risks are, that we don't know the time scale, dismissing the concerns is dangerous. What we're trying to do is to push it forward in the respectable scientific community," he said.
While Price said the exact nature of the risks is difficult to predict, he said that advanced technology could be a threat when computers start to direct resources towards their own goals, at the expense of human concerns like environmental sustainability.
He compared the risk to the way humans have threatened the survival of other animals by spreading across the planet and using up natural resources that other animals depend upon.
Price is co-founding the project together with Cambridge professor of cosmology and astrophysics Martin Rees and Jann Tallinn, one of the founders of the internet phone service Skype.
The university said Sunday the center's launch is planned next year.