Nova Spivack on the Semantic Web future

Nova Spivack has some interesting thoughts on the Semantic Web future. His company Radar Network, a software company funded by Paul Allen’s Vulcan Capital, is developing a new semantic platform and online service for group communications and collaboration.

In his essay Minding the Planet: the Meaning and Future of the Semantic Web, Spivack discusses important issues that surround the present Semantic Web development and the future Web.

But what is the Semantic Web, and why does it matter, and how does it enable collective intelligence? And where is this all headed? And what is the long-term fa future going to be like? Is the global mind just science fiction? Will a world that has a global mind be good place to live in, or will it be some kind of technological nightmare?

Spivack sees the Semantic Web vision is more than just building a new Web that will help computing machines to be intelligent. He believes that the vision is to create an intelligent Web that will provide technologies to enable people, groups, organizations and communities to be smarter.

In short, the Semantic Web aims to make software smarter, not just for its own sake, but in order to help make people, and groups of people, smarter. In the original Semantic Web vision this fact was under-emphasized, leading to the impression that Semantic Web was only about automating the world. In fact, it is really about facilitating the world.

I agree with Spivack on this point. If Web technology is to increase people’s productivity, any new technology that we create should aim to solve problems for people and not for computers.

Spivack believes that semantic metadata will play an important role in the future Web. In particular, he sees folksonomy tagging and semi-automatic concept extractions as key enabling technologies for the Semantic Web 2.0.

By marking up information with metadata that formally codifies its context, we can make the data itself “smarter”. The data becomes self-describing. When you get a piece of data you also get the necessary metadata for understanding it. For example, if I sent you a document containing the word “sem” in it, I could add markup around that word indicating that it is the word for “mind” in the Tibetan language.

Instead of trying to make software a million times smarter than it is today, it is much easier to just encode more metadata about what our information means. That turns out to be less work in the end. And there’s an added benefit to this approach — meaning exists with the data and travels with it.

Spivack is a believer of emergence. One way to create a smart Web is to rely on the emergence of collective intelligence — the combined knowledge of information contributed by users of the Web. He speculates the creation of these Web sites and points to Flickr and Digg as examples that encourage the emergence of collective intelligence.

When you look at them close-up they appear to be just like any other Web site, but when you look at what they are doing as a whole — these services are thinking. They are learning, self-organizing, sensing their environments, interpreting, reasoning, understanding, introspecting, and building knowledge. These are the activities of minds, of intelligent systems.

Flickr and Digg also exhibit intelligence. Flickr’s growing system of tags is the beginnings of something resembling a collective visual sense organ on the Web. Images are perceived, stored, interpreted, and connected to concepts and other images. This is what the human visual system does. Similarly, Digg is a community that collectively detects, focuses attention on, and interprets current news. It’s not unlike a primitive collective analogue to the human facility for situational awareness.

The future of the Semantic Web looks very exciting. If you want to be a pioneer of the Semantic Web 2.0, here are few tips from Spivack:

Representing individual knowledge. The first step is to make individuals’ knowledge accessible to themselves. As individuals become inundated with increasing amounts of information, they will need better ways of managing it, keeping track of it, and re-using it. They will (or already do) need “personal knowledge management”.

Connecting individual knowledge. Next, once individual knowledge is represented, it becomes possible to start connecting it and sharing it across individuals. This stage could be called “interpersonal knowledge management”.

Representing group knowledge. Groups of individuals also need ways of collectively representing their knowledge, making sense of it, and growing it over time. Wikis and community portals are just the beginning. The Semantic Web will take these “group minds” to the next level — it will make the collective knowledge of groups far richer and more re-usable.

Connecting group knowledge. This step is analogous to connecting individual knowledge. Here, groups become able to connect their knowledge together to form larger collectives, and it becomes possible to more easily access and share knowledge between different groups in very different areas of interest.

Representing the knowledge of the entire web. This stage is still in the distant future, but at this point in the future we will begin to be able to view, search, and navigate the knowledge of the entire web as a whole. The distinction here is that instead of a collection of interoperating but separate intelligent applications, individuals and groups, the entire web itself will begin to function as one cohesive intelligent system. The crucial step that enables this to happen is the formation of a collective self-representation. This enables the system to see itself as a whole for the first time.

I invite you to read Nova Spivack’s essay in its entirety (PDF). Not that his essay is an exact road map for the future Web — no one is really at predicting the future, but it does raise many interesting and important issues that researchers and developers must consider.

Additional blog discussions: