Geospatial Semantic Web Blog

Downloading files is one of the most frequently performed task on the Internet. Today there are many protocols for downloading files — e.g., HTTP, FTP and bittorent. Different sites (e.g., Linux distributions, OpenOffice) offer downloads in some or all those protocols. Metalink is a new XML-based markup language and technology for unifying file downloads across multiple protocols. Digging deeper into the Metalink format representation, I find some surprises — the use of tagging and expressive content description.

Problem

There are at least two problems existing in the current file downloading scheme.

One, it’s difficult for the end users to discover the most speedy way to download a file. Sites often list dozens of mirror servers for file downloads. Depending on the workload of each server, the download time can vary greatly. For the users, it’s not always easy to find out which mirror server can offer the shortest download time.

Second, there is no unified format representation for descrbing file downloads. For protocols such as FTP and HTTP, it’s not usually easy to say things like “from this URL, you can download a copy of OpenOffice version 2.1.0 for the Windows platform.” Although some P2P protocol such bittorent allows the explicit description of file downloads, but such description is not usually understood by the non-P2P clients (e.g., Web Browsers).

What’s Metalink?

In the Metalink 3.0 Specification, Metalink is defined as the follows.

Metalinks bundle the various ways (FTP/HTTP/P2P) to acquire files into one format for easy content distribution/Electronic Software Distribution (ESD). This can increase download performance, reliability, usability, & efficiency. Any program that downloads files could potentially use it. A Metalink is a Multi Method Metalinker (MMM or M3talinker) document. MMM is a dialect of XML and MMM files must conform to the XML 1.0 specification. “MMM” might also look like “WWW” mirrored.

Solutions

Metalink solves the first problem — how to find the most speedy way to download a file — by grouping different download protocols into one protocol. This enables Metalink clients to automatically switch between different mirror servers without explicit user instructions.

The following is a code snippet of a Metalink description for the OpenOffice downloads:

In the above example, not only multiple server information is described but also meta-data (e.g., the location of the server, document tags and license). Although this information is not in RDF, but it’s clear that it provides expressive description about file downloads. I imagine this information can be consumed by Metalink clients to build up smart search capabilities and maybe recommend files to users.

Remarks

I’m quite excited about Metalink. While there is no guarantee that Metalink will revolutionize Internet file downloads, but its use of markup languages to describe information on the Web and its attempt to shield users from heterogeneous download protocols are signs of the emerging Semantic Web.

Resources:

• Making Metalinks — an article on Metalink and how to use it.
• Who uses Metalink? — a list of sites/projects that support Metalink
• Metalink Generator — a Web-tool for generating Metalink files