Joy warns of tech Armageddon

Daily Newsletters

Sign up to ZDNet UK's daily newsletter.

NEWS
Citing "Unabomber" Theodore Kaczynski and other information age Cassandras, Sun Microsystems' co-founder and chief scientist, Bill Joy, is now warning of the potential dangers in the computer technologies he helped create. In an article scheduled for publication on Tuesday in the April edition of Wired magazine, Joy cautions that the convergence of genetic engineering and computer technology could pose a very real threat to humanity and the ecosystem. Joy quotes from a passage of Kaczynski's Unabomber "Manifesto", which posits a scenario where computers become sufficiently sophisticated to make their own decisions, and where any human control over the systems devolves to a "tiny elite" who will either find means to eliminate the masses or -- "if the elite consists of soft-hearted liberals" -- tend to their physical needs while eliminating their free will through biological or psychological engineering. "These engineered human beings may be happy in such a society, but they will most certainly not be free," Kaczynski wrote. "They will have been reduced to the status of domestic animals." Joy is quick to decry Kaczynski's bombing campaign as a "criminally insane" act. Nevertheless, Joy writes, "as difficult as it is for me to acknowledge, I saw some merit in the reasoning in this single passage. I felt compelled to confront it." In Joy's view, both past precedent and current technology point to the danger that human creations may present. For example, the emergence of antibiotic-resistant bacteria may be merely a foretaste of "Borg-like disasters" that could occur if the latest scientific advances spin out of control. "Accustomed to living with almost routine scientific breakthroughs, we have yet to come to terms with the fact that the most compelling 21st century technologies -- robotics, genetic engineering and nanotechnology -- pose a different threat than the technologies that have come before," Joy writes. "Specifically, robots, engineered organisms and nanobots [molecular-sized, intelligent machines that can be programmed to alter microscopic structures such as DNA] share a dangerous amplifying factor -- they can self-replicate. A bomb is blown up only once, but one bot can become many, and quickly get out of control. Much of my work over the past 25 years has been on computer networking, where the sending and receiving of messages creates the opportunity for out-of-control replication. But while replication in a computer or a computer network can be a nuisance, at worst it disables a machine or takes down a network or network service. Uncontrolled self-replication in these newer technologies runs a much greater risk: A risk of substantial damage in the physical world." According to Joy, current advances in molecular electronics mean that by the year 2030, "we are likely to be able to build machines in quantity a million times as powerful as the personal computers of today", and imbue them with human-level intelligence. "In designing software and microprocessors, I have never had the feeling that I was designing an intelligent machine," Joy writes. "The software and hardware is so fragile and the capabilities of the machine to 'think' so clearly absent that, even as a possibility, this has always seemed very far in the future. But now, with the prospect of human-level computing power in about 30 years, a new idea suggests itself: I may be working to create tools which will enable the construction of the technology that may replace our species. How do I feel about this? Very uncomfortable. Having struggled my entire career to build reliable software systems, it seems to me more than likely that this future will not work out as well as some people may imagine. My personal experience suggests we tend to overestimate our design capabilities." Joy offers some qualified words of optimism that humanity will be able to alter this course by establishing curbs modelled after global governments' handling of weapons of mass destruction. "We are being propelled into this new century with no plan, no control, no brakes. Have we already gone too far down the path to alter course? I don't believe so, but we aren't trying yet, and the last chance to assert control -- the fail safe point -- is rapidly approaching. We have our first pet robots, as well as commercially available genetic engineering techniques, and our nanoscale techniques are advancing rapidly. We are getting a belated start on seriously addressing the issues around 21st century technologies and further delay seems unacceptable." What do you think? Tell the Mailroom and read what others have to say.

Post your comment

In order to post a comment you need to be registered and logged in.

You can also log in with Facebook. Log in or create your ZDNet UK account below

  • Login

Will not be displayed with your comment

By signing up for this service, you indicate that you agree to our Terms and Conditions and have read and understood our Privacy Policy. Questions about membership? Find the answers in the Community FAQ

Get ZDNet UK's daily newsletter

Enter your email address to sign up

ZDNet UK Live

JCB33

How dare film makers, artists or anybody that invests in creativity stop us pirating their works for free. I want to be able to walk into my local...

4 hours ago by JCB33 on ACTA stumbles in Germany
Moley

@GrueMaster. I prefer horses for courses rather than one size fits all. I, and I suspect most other computer users, do not really wish to have...

6 hours ago by Moley on A tale of two distros: Ubuntu and Linux Mint
greycynic

The product that scares me every time I have to use it is the Office 2007 version of Excel. The first bug that I found was applying the median...

6 hours ago by greycynic on Ten flawed products that derail productivity
GrueMaster

Nice review and very informative. One thing I'd like to add (in reply to whs001's 1st question), the main reason to have the same interface from...

7 hours ago by GrueMaster on A tale of two distros: Ubuntu and Linux Mint
Frederick Wrigley

I'be been using Mint 12 since the RC came out, and I am far more happy with the Cinnamon, the Mate, and, yes (with extensions), theGnome 3...

8 hours ago by Frederick Wrigley via Facebook on A tale of two distros: Ubuntu and Linux Mint
bdantas

Excellent article. One small correction, though--although a fresh installation of Linux Mint 12 will, indeed, provide the user with a version of...

9 hours ago by bdantas on A tale of two distros: Ubuntu and Linux Mint
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

9 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Alan Ralph

In related news, the ISPs club together to get the members of the Home Affairs Select Committee (ya goofed on that part, ZDNet UK) copies of "The...

9 hours ago by Alan Ralph via Facebook on MPs urge ISPs to take down terrorist material
Moley

For Gnome 2 die-hards, it is possible to add icons to the bottom panel (or top top panel, if you prefer) which provide the exact Gnome 2...

10 hours ago by Moley on A tale of two distros: Ubuntu and Linux Mint
ramwellian

Your comments would seem pretty naive and immature. Your 'solution' appears to be, "gee, let's all just give in to the hackers and give them...

10 hours ago by ramwellian on Cloud computing security: no more oxymoron?
BugStalker

"Interesting thought ... If you installed Win7 as a dual boot on a machine that previously only had Linux, and it wrecked your Linux installation,...

11 hours ago by BugStalker on Windows 7 Declares War on GRUB
whs001

This is an excellent summary of Ubuntu and Mint and the interface differences between them. Most such articles take a very partisan position for...

11 hours ago by whs001 on A tale of two distros: Ubuntu and Linux Mint
Moley

@ewallace. Not so clear. Anyone can obtain the text, for example from here http://www.ustr.gov/webfm_send/2379. I support ACTA so long as it and...

11 hours ago by Moley on ACTA: Facts, misconceptions and questions
45283

I think WinRT is fantastic. I just wish it was an option for people that didn't want to go through Microsoft's App Store with its attendant...

14 hours ago by 45283 on Why Windows 8 needs architectural hygiene for WOA
Burn-IT

Nine people? £30m? Who's back pocket is that lot going in? And IF they say it is for new buildings, what about all the ones the government has...

15 hours ago by Burn-IT on Police set to launch three £30m e-crime hubs
ewallace

Just to be clear, nobody knows what is in the text of ACTA, here is a photograph of the text of ACTA http://twitpic.com/8h9iju as submitted to the...

15 hours ago by ewallace on ACTA: Facts, misconceptions and questions
fgvrg56

Unfortunately main issue is that ASUS is refusing to accept that they make some mistake on this version of asus Transformer prime. 1 - GPS sensor...

16 hours ago by fgvrg56 on Asus Eee Pad Transformer Prime Wi-Fi & GPS problems?
Ben Woods

@Marcus A fair question. Just talked with Archos which said it was working on an announcement for next week....

18 hours ago by Ben Woods on Archos confirms G9 Ice Cream Sandwich update schedule
Marcus Karlsson

Any update on this, considering the claimed "first week of February"?

19 hours ago by Marcus Karlsson via Facebook on Archos confirms G9 Ice Cream Sandwich update schedule
apexwm

Bill Goodrich : Just as al_langevin pointed out, with Windows Server 2008 there is no Services for Macintosh anymore. It's gone, not available....

1 day ago by apexwm on Windows Server 2008 drops the ball for Mac compatibility