Dev News Daily ENDE

curl turns 30: from a 100-line HTTP fetcher in 1996 to a library in billions of installs

On 11 November 1996 a Brazilian student, Rafael Sagula, put the source of a small HTTP download tool online. It was called httpget 0.1, it ran to a little over a hundred lines of C, and it took raw IPv4 addresses rather than host names or URLs. Daniel Stenberg, who found the code a few days later, marked the thirtieth anniversary of that release on his blog this week, and the post is a compact history of how the tool became curl.

Stenberg did not set out to build a transfer library. He wanted exchange rates for an IRC bot called Dancer that he and friends ran in their spare time, so that people in the channel could convert prices between currencies. httpget fetched the rate page once a day. He fixed the bugs he hit, added name resolution and mailed diffs to Sagula, which was the normal way to share changes at the time. Sagula showed little interest in maintaining it, so Stenberg ended up as maintainer.

The name changed twice for practical reasons. When a gopher server turned out to offer more currencies than the web page, gopher support went in and the tool became urlget; FTP followed. Early in 1998 upload support arrived, both FTP upload and HTTP POST, and the name was wrong again. Stenberg wanted something with URL in it that was short to type, and settled on curl, which can be read as see URL. curl 4.0 shipped on 20 March 1998 with about 3,300 lines of code, 24 command-line options and three protocols.

The library came two years later. In the summer of 2000 Stenberg refactored the internals and the first libcurl appeared in August that year. PHP was among the first adopters and made libcurl its default engine for internet transfers. Version control arrived late as well: the project only moved fully to CVS in September 1999 and switched to git in 2010.

The post sets the scale against the period. A Swedish household connection in 1996 averaged 30 kilobits per second; today the average is about 200 megabits. There were roughly 36 million internet users and 250,000 websites then, against six to seven billion users and around a billion sites now.

curl turns 30: from a 100-line HTTP fetcher in 1996 to a library in billions of installs
curl turns 30: from a 100-line HTTP fetcher in 1996 to a library in billions of installs — Dev News Daily

Why it matters

Stenberg puts libcurl at an estimated thirty billion installations, and says plainly that the figure is rough because most users never report that they ship it; a single popular app adding or dropping the library moves the total by billions. His point is that the number changes little about the work. The same small group of people still maintains the code in much the same way, which is a useful reminder of how much of the stack rests on a handful of maintainers.