Monthly Archives: January 2013

Just how big are porn sites?

Image

It is a truth universally acknowledged, that a person in possession of a fast internet connection must be in want of some porn.

While it’s difficult domain to penetrate — hard numbers are few and far between — we know for a fact that porn sites are some of the most trafficked parts of the internet. According to Google’s DoubleClick Ad Planner, which tracks users across the web with a cookie, dozens of adult destinations populate the top 500 websites. Xvideos, the largest porn site on the web with 4.4 billion page views per month, is three times the size of CNN or ESPN, and twice the size of Reddit. LiveJasmin isn’t much smaller. YouPorn, Tube8, and Pornhub — they’re all vast, vast sites that dwarf almost everything except the Googles and Facebooks of the internet.

While page views are a fine starting point, they only tell you that X porn site is more popular than Y non-porn site. Four billion page views sure sounds like a lot, but it’s only when you factor in what those porn surfers are actually doing that the size and scale of adult websites truly comes into focus.

We’ll start by laying the ground work, and then on the second page we have some real world figures from YouPorn, the second largest porn site on the web. If you like, take a moment to try and estimate the amount of traffic that YouPorn handles every second. Let us know in the comments if your guess is anywhere near.

Scale

Xvideos in Ad PlannerThe main difference between porn and non-porn sites is the average duration of a visit: For a news site like Engadget or ExtremeTech, an average visit is usually between three and six minutes; enough time to read one or two stories. The average time spent on a porn site, however, is between 15 and 20 minutes.

Then you need to factor in that most websites are predominantly text and images, while the largest porn sites push streaming video. When you load the ExtremeTech home page, you’re talking about a couple of megabytes, and then maybe 500 kilobytes if you load an article. When you stream porn, assuming a low resolution of 480×200, you’re looking at around 100 kilobytes per second — which, over 15 minutes, is around 90 megabytes.

Then you need to multiply 90 megabytes by the number of monthly visits — which is around 350 million for Xvideos. This comes to around 29 petabytes of data transferred every month, or 50 gigabytes per second. To put this into comparison, your home internet connection is probably capable of transferring a couple of megabytes per second, which is about 25,000 times smaller.

In short, porn sites cope with astronomical amounts of data. The only sites that really come close in term of raw bandwidth are YouTube or Hulu, but even then YouPorn is something like six times larger than Hulu.

Infrastructure

Serving up videos requires a lot more resources than plain text and images, in terms of storage, CPU cycles, internal I/O, and bandwidth.

Backblaze Storage Pod partially filled caseWhile it obviously varies from site to site, most adult sites will probably store in the region of 50 to 200 terabytes of porn. This is quite a lot for a website (only something like Google, Facebook, Blogger, or YouTube would store more data), but in a world where 2TB drives are cheap and plentiful, this isn’t ultimately a very large amount. Last year we wrote about a Backblaze storage pod that can store 135TB in a 4U case, for just $7,400.

CPU cycles and I/O will be a function of the bitrate of the streaming video and the number of page views. First the porn site has to serve up a dynamic, searchable database of thousands of videos, and then, when someone clicks on a video, that file needs to be read from a hard disk and streamed over the internet. If you’ve ever transferred a lot of big files over a local network (i.e. stressed both your hard drive and Ethernet port) you will know how taxing this is.

Actual hardware requirements are almost impossible to derive (they’re not publicized), but in the case of a large porn site we’re probably talking about racks of quad-CPU servers, gigabit switches, and load balancers. Software-wise, most large porn sites will use a very-high-throughput database such as Redis to store and serve videos, and a light-weight HTTP server like Nginx to serve up the web pages.

Finally, bandwidth. Referring back to our Xvideos example (based on an Ad Planner estimate), a large porn site will have to have enough connectivity to serve up 50 gigabytes per second, or 400Gbps. Bear in mind this is an average data rate, too: At peak time, Xvideos might burst to 1,000Gbps (1Tbps) or more. To put this into perspective, there’s only about 15Tbps of connectivity between London and New York.

There are only so many ways of coping with this much traffic: You set up your own data center, rent a few racks in a very large data center, or use a cloud provider like Amazon AWS or Microsoft Azure.