Late last summer, the popularity of a music-sharing phenomenon called Napster sent cold shivers down the collective spines of music publishers everywhere. If you had read the story of Napster's battle with 'Big Music', you may have thought it had more to do with the future of publishing and the protection of intellectual property, than with the next step in the evolution of the internet.
A 'civilisation-altering' event for the digital-media industry but still a primitive example of an emerging technology, Napster's shadow extends far beyond questions of copyright and the pirating of popular music. A new Pandora's box has been opened, a paradigm shift in technology, which suggests that a globally distributed framework of personal computers, now called the Grid, can compete with or even replace the powerful network of centralised servers that we recognise as the world wide web.
Grid computing, more usually described as 'peer to peer' (P2P) or distributed computing, may change the way in which we interact with the growing mountain of information that surrounds us. It will influence the way software is designed, and its touch is already being felt through the arrival of products such as Sony's PlayStation 2 and the vision behind Microsoft's 'Dot Net' strategy.
Indeed this vision may pull the PC back from the brink of redundancy. On Thursday Microsoft issued its first prof its warning in 10 years. Analysts inferred an irreversible decline in the PC software market and Microsoft shares lost a third of their value.
But P2P offers the technology to use the spare capacity of home computers.
In fundamental terms, the Grid represents a devolution of the internet's own form of central government, a redistribution of power among millions of connected devices, creating what is effectively a global supercomputer from the network itself.
In a flash of genius, the scientific research institute Search for Extraterrestrial Intelligence (Seti) enlisted the help of 2 million PC users to analyse daily radio spectrum searches, having found the cost of using supercomputers prohibitively expensive. Interested PC owners downloaded Seti's analysis software from the internet and installed it as a screen saver. When the user wasn't working on the PC, the Seti@home program accepted a proportion of the organisation's workload from the internet and processed the data, passing results back to Seti's main system in California. In practice, Seti created a huge supercomputer by using its volunteers' idle processor time.
Napster, which claims 25 million users, has a similar service, Aimster, which uses AOL to access a network of 120 million people. The files they share are most commonly compressed (MP3) music, exchanged on a 'person to person' basis rather than through access to a central internet server; although the existence of a centralised directory on a Napster server was the excuse the music industry needed to take Napster to court. Like Seti@home, this is a P2P network.
With millions of people using the internet every day, and the wider availability of high-bandwidth connections, many types of device can conceivably work together as a single distributed system. The whole becomes greater than the sum of its parts, much like bees in a hive, capable of contributing all or part of their resources to a much greater purpose.
This prompted Intel chief technology officer Pat Gelsinger to comment: 'Peer-to-peer computing could be as important to the internet's future as the web browser was to its past.' Indeed, the company, according to a report in Red Herring magazine, claims to have saved $500m 'by leveraging the power of its own worldwide network of personal computers'.
Intel views the PC-centric nature of the Grid as an opportunity for the company to maintain its dominance. Other server manufacturers, whose businesses are based on very large and powerful servers, may not be so happy at the idea of a network of millions of PCs replacing larger computers.
The appeal of P2P networking is that as the Grid expands, the number of connections between users grows exponentially, creating a vast and almost brain-like shared network.
But P2P, which really involves two young technologies, the sharing of digital content and the sharing of computers, has still to reach the point where it can leap from being an untidy commercial threat to old-econ omy interests to a sound economic proposition for a networked society.
As an investment prospect, P2P offers two emergent market opportunities: resource allocation, where companies may rent network bandwidth or processor time on the public's PCs, and shared media, digital audio and multimedia distribution. But the challenges are great, the returns uncertain and overall investment in the sector remains modest - reportedly below $100m. Grid computing requires that special software and intelligent agents be installed on participating devices, allowing it to act as both client and server, and making it capable of initiating and responding to search requests that can swiftly ripple through all other shared devices linked to it.
Internet search engines can be notoriously ambivalent when it comes to refined data searches: in most cases, the information retrieved is not real-time and only as good as the last data cache on the web search engine. Activity from start-up companies is now being focused on creating software that can work in and manage transactions on the scale of a P2P network of thousands, if not millions, of PCs. This relies on these PCs being connected to the Grid, sharing their local processors and their 'public' files and directories. As an example, each PC in one school could be connected to the PCs in every school in the country, creating an enormously powerful educational supercomputer in the process. In this way, any information on the network becomes a real-time shared resource.
Before the Grid can make even a modest assault on the infrastructure of the Internet, two distinct problems have to be overcome: privacy and bandwidth. With cybercrime now a rampant problem, the internet's more centralised model does at least offer some protection from a direct assault on a PC. However, unless security can be guaranteed, people will be reluctant to adopt an 'open-house' approach to the future of personal computing.
With the internet becoming the delivery mechanism for every conceivable form of digital media and entertainment content, bandwidth is constantly playing catch-up with demand. Grid computing over a P2P network of tens of thousands, if not millions of computing devices, all passing data between themselves will demand huge amounts of reliable bandwidth.
Just as we were becoming comfortable with one view of technology and the evolution of the internet, another, larger wave of more radical change may be about to wash over society. The Grid's commercial potential still remains unclear and a host of issues need resolving.
A universal micro-payment system is also required, so that should a large research company wish to hire the number-crunching power of 10,000 volunteer PCs overnight, a route for the owner to receive a tiny transactional amount in return becomes available.
As the network becomes the computer, the principles of the Grid, as expressed by one analyst, Alan Zeichick, may point to a radically different future: 'Millions of participants, no central servers, no search engines, no registration, no index, no database, no identification - and nobody for the lawyers or governments to sue or negotiate with.'