Wednesday, December 14, 2011

The most popular content on the web in the last two years

As I speak to people, I often ask what they think creates the most Internet traffic.  The answers are often illuminating, showing that usually we don't really know - like roads to where we want to go and electricity usage we just assume that it is there until there's a problem and don't worry so much about what is really happening and what generates traffic.

I have asked the question a lot in the last few weeks as I travelling around EMEA for the Blue Coat security roadshows, here's a few answers I got and then the actual answer to a specific question you can think about yourselves:

What generated the two highest peaks of traffic in London in the last two years.

Some people have commented on the type of traffic, the applications and protocols.  This has been well-documented already, my favourite image for this is over a year old, but still valuable - the graphic from Wired magazine showing traffic patterns over the last few years, clearly streaming is number one and growing fastest.

So, let's ignore protocols and applications, but what events or activities have actually generated the most traffic?  The incorrect answers I can remember are:

1. The wedding of prince William and Kate Middleton
2. The Arab Spring
3. The Japanese earthquake and tsnami
4. The death of Michael Jackson
5. The death of Obama Bin Laden

Each of the above were major news items and some generated quite a lot of web trafic, especially a lot of video traffic - but as major traffic generators they are eclipsed by two other events.  Perhaps the wedding would have generated more, but as Great Britain has a national holiday so those that wanted to watch it live were watching on TV and not across the web.

So, have you thought yet?  The type of material probably is large, very popular (and not just to news junkies) and probably time-critical.  Let me help you with a graphic from LONAP that gives you a couple of clues.

Firstly, what is LONAP?  LONAP is a Layer-2 Internet Peering Point in London owned and run by over 100 ISPs - it usually peaks at around 20Gbps and the daily and weekly (MRTG) traffic graph sits here:

This clearly shows the usual pattern of traffic, showing the days busier than the evenings and weekends quiters than during the week.  (I remember when evenings and weekends were the busiest, showing how important the Internet is to businesses, but perhaps that's all very obvious).

Anyway, these graphs are constantly updated, so this one I can going to paste in, from the page that goes a bit further back - now does this help you answer my question?:

The Answer to My Question

This gaph starts from the end of April 2010 and you can see three peaks - The largest is at the end of June 2010, the second largest in mid-October 2011 and a smaller one just at the beginning of December 2010.

So - June 2010.  This was the 2010 Football World Cup, the contest took place in South Africa and some of the matches were during the working day in the UK.  As you've seen from my previous posts, there were some network problems in individual organisations, though the Internet itself coped admirably.  No surprise really, major sporting events always drive a lot of traffic - the fans want the immediacy of the game, there's much less point in watching it after you have heard the result.

Perhaps October 2011 is more difficult to explain, though it is more recent.  Was there a major sporting event?  News?  Music?  er .....

Someone said the death of Steve Jobs.  No - terrible news of course and it generated lots of traffic, lots of news stories, but actually Steve had a hand in the traffic a few days later.

Yes, the launch of iOS5.  On October 12th, every iPhone, iPad and iPod Touch that connected to the Internet offered the user a download of iOS5.  There was also a new version of iTunes for iOS, Lion and PCs and most other iOS applications had a new version too.  Huge amounts of traffic were generated over a short period of time - after probably 48 hours most devices/applications had been updated and the traffic patterns returned to normal.

So, something to think about.... 

Traffic is often generated by time-critical happenings, some we can plan for (the next great sporting event), some we won't kow about (Apple, Microsoft, Adobe updates), some are completely unplanned (death of a celebrity, weather, disasters).  In addition, some are global (iOS), some are local (World Cup traffic highest in countries taking part, of course) and some may be particular to your business (CEO announcement, competitor, new product/training videos).

From a technical/network point of view - are you ready for these sorts of peaks, how much headroom do you have and are you implementing technologies such as WAN optimisation to improve your total throughput?

Meanwhile, I have one more question.... What happened at the beginning of December 2010 that caused huge amounts of Internet traffic in London?  Was it everyone watching the weather forecasts and updating each other on #UKSnow on Twitter or did something else happen that I have forgotten?

Let me know what you think.

Wednesday, November 30, 2011

Marketing on Cloud 9 - Yes please

Marketing on Cloud 9 – yes please

Where would we marketers be without LinkedIn, Google Analytics, Eloqua, SalesForce, Facebook and Twitter? We’d be back in the dark ages, of course, with our competitors eating our lunch!

The way that marketing technologies are used and consumed has evolved considerably in recent years, with cloud computing and web technologies in particular taking off as a means to access information, applications, to get feedback from our customers and deliver new campaigns. So what else can business cloud-computing offer and why should it be embraced by marketers? Equally, what challenges should users be aware of when considering cloud applications and how can these be mitigated?

Put simply, business cloud computing means accessing applications via the internet so that vast amounts of computing resources will reside somewhere else and not on local PCs or servers. This, of course, means that you’re no longer tied to fixed servers, software packages and expensive management costs. From applications for CRM to those for budgeting, planning and collaboration, the cloud delivery model is playing a very central role in the marketers’ toolkit.

So does the way we consume applications really matter for the marketer? The real benefits relate to cost and efficiency which should be welcome news to anyone faced with increasing pressures on budgets. For the business as a whole, it means outsourcing the burden of maintaining servers and applications; as well as saving time and money through replacing the capital expenditure traditionally spent on IT infrastructure with operational expenditure.

For marketers, it also presents an opportunity for improved and streamlined business processes, with the creation and availability of reports, campaigns, customer lists and hefty documents and images and media on demand. With marketing teams now increasingly operating in a 24/7 always-on environment, the cloud can be a route to more efficient collaboration; a crucial factor in an industry reliant on speed and communication. It enables teams to share valuable customer data or analytics so that everyone - be they in different time zones or regions - has access to the same information in real time.

The cloud also provides scalability; you’re no longer tied down to expensive software packages, but instead are able to select the most appropriate on-demand, pay-per-use technologies to fit the particular projects and campaigns. Depending on the size of the project, marketing teams can opt in or out, choosing whether to scale up or down their IT infrastructure.

This all sounds like great news, however there are considerations that need to be made as cloud applications becomes more widespread. One such issue is that of the performance challenges that can be associated with cloud-based applications, including service speeds, response times and increased bandwidth consumption. There’s nothing more frustrating than waiting to access an application that is slow to respond or inputting data that takes an age to refresh. This of course means frustration for the user, but can also have a real financial implication – equating to loss of productivity. But we marketers need to access data quickly to make the most accurate and informed decisions.

We marketers are often ahead of the game in making demands on the IT infrastructure; we always have the largest graphics files or need that HD video downloaded in minutes. Some people equate performance with simple bandwidth and many of us have been sent away by the IT department saying that it cost too much to increase the bandwidth to the office. Thankfully, there are now solutions available that can address the performance challenges of applications and content delivered via the cloud and they are not by adding bandwidth, these WAN optimisation solutions make better use of the bandwidth already available and can multiple the throughput tens, hundreds or even thousands of times, depending on the material being transmitted.

In the past, solutions have been unable to optimise applications based in the public cloud where the infrastructure is not owned or managed by the customer. However, new ‘one-sided’ optimisation technologies that only need to be installed at the user’s end of the link (office based) mean that businesses can now accelerate content and applications from the public cloud without having an appliance located in the cloud data centre where that particular service is hosted. So, you can go to your IT department and help educate them on your needs and give them a guide to the solutions at the same time.

Whilst the cloud promises much, the end-user experience is key to its success. The ability to access critical applications at optimal speed, with the assurance that the corporate network can cope is critical. In this way marketers can embrace cloud computing and use critical applications with the speed and ease to experience real business benefits

Friday, September 16, 2011

The Internet Promised Disintermediation - Here it is...

Over 15 years ago, I remember 3Com and U.S. Robotics people including myself talking about the Internet bringing disintermediation - the removal of intermediaries in the supply chain - the ability for the producer and the ultimate end user doing business together with no channel partners in the middle.

Funnily enough though, if you look at some of the greatest Internet successes; Amazon, eBay and (at a stretch) Google - they are the epitome of intermediaries - great resellers or putting customers and suppliers together and even Google's search results are just a list of data on other people's web sites.  Then  you can look at the price comparison sites and ticket and trip/holiday sites and often they are intermediaries too.  Skype - joins together people, Spotify, just another way to consume music.

So, were we wrong?  Were we fooled?  Did we expect more seismic changes than we got?

I'd say that the change is happening, but only in some industries and at a slower pace than we technologists thought (no surprise there, it always happens like that).

However, nowadays many musicians sell some of their material direct to their fans - perhaps only the odd live album for the most successful artists, but it is certainly keeping older artists going who are still making interesting music but who's fan base is smaller.

Of course, it is those purchases that are most easily digitised that have the greatest opportunity - so the flurry of announcements from movie studios is interesting and no doubt very scary if you are one of the intermediates.

As an example, look at this link from Miramax.  Famous movies available for the equivalent of $3 each.  Many times cheaper than going to the movies (and you don't have to listen to teenagers chatting/texting), less hassle than finding a Blockbuster nearby and lower price (at the moment) than LoveFilm or equivalent general film download services.  If this is successful, what space the video shop? The retailer offering you downloads? Miramax hold the cards if they want to destroy the intermediaries.

So, I guess the future is coming, just not quite as fast as we thought, I certainly wouldn't be opening a video store at the moment.

Wednesday, July 27, 2011

World Cup 2010 - How did your network do?

Someone asked me yesterday about another old article I wrote around the network problems seen during the 2010 World Cup.  They are rightly worried about the performance problems we should expect during next year's London Olympic Games.
I actually wrote two articles - one before the event and one afterwards.  The one before the event is currently available online: 
... the one after the event was over is reprinted below.

How well did your network play during the World Cup?

As predicted, the 2010 FIFA World Cup was the biggest global event in web history.  More people watched the matches, more tweets were posted and more pages viewed and updated than any previous event.  Of course, this huge spike of traffic caused a number of business problems; reminding us of the inexorable rise in traffic, the blurring between work and personal usage, and most importantly that IT management need to plan for the next “big thing” to ensure that business can continue during popular events.

Akamai claimed a peak of 1.6million simultaneous streams, many in HD, and many broadcasters around the world delivered twice the previously-highest peak traffic. Just in the UK, the BBC delivered 800,000 streams during the England vs. Slovenia match alone and total UK Internet usage increased by over 30%.  People watched on their PCs, through their mobile phones and iPads, at home, while travelling and in the workplace.  The BBC statistics for June 2010 showed that there were 9.7million requests for live simulcast content in June 2010, an increase of 26% over the previous month and around a 500% increase on a year ago [source BBC iStats].  This reminds us of the growing expectation that live TV can be watched online and that if something is considered newsworthy that users will do this, even during the working day.

As with car traffic to popular events, the traffic problems are in the last few miles.  It wasn’t the main Internet highways that suffered problems, but the physical bottlenecks along country roads and into car parks were replaced by the final connection to the businesses or sometimes the initial connection to the broadcaster.  Some businesses found that their own Internet connections and links to branch or remote offices using their Wide Area Network (WAN) were overloaded by World Cup traffic, meaning that other data couldn’t be sent or received and business-critical applications came to a halt.

During the initial matches, a number of broadcasters had underestimated the demand – as their servers failed to deliver the quantity of requests made, leading to outages and poor quality video – Twitter and blogs were full of complaints.  It’s clear that a significant demand came during business hour when employees were presumably at work. By the end of the tournament it seemed most of the complaints had subsided or perhaps viewers went back to watching on their TVs – though the later-timed matches perhaps helped the broadcasters out too.

Any popular event will entice the scamers and malware writers out from their dark corners, and the World Cup was no exception. Eight out of the top ten spam messages during June were related to the World Cup, including countless phishing ploys, and there were many Web pages trying to entice people to pay for promised online coverage (often in HD). Scam news articles promised behind the scenes footage that led to the common ploy of “update your codec here” attempts to surreptitiously download malware.

The main issue was the clearly the impact on business networks – there were stories of network traffic failures, followed by rapid emails asking everyone in an organisation to stop watching the World Cup and people saying “if you can’t beat them, join them” as they downed tools for an hour and a half during a particular match.

Just during a 30 minute period I picked up the following tweets, some about business traffic and some just complaining that watching a match was impossible:

AgentOwen: Our internet usage at the office has gone way up during World Cup.  We just got a spanking as it’s slowed down our network.
Dave: Received email from IT “Don’t watch the World Cup – we’ve got business to do” – fine, I’ll go down the pub.
Speedvegan: Note to self: Do not schedule any releases while the US is playing in the World Cup. Network slows to a halt.
Flokemon: Ingerland playing, USA playing, corporate network slows down to a crawl, can I get a stream working eeeekk
Epheramaldog: Ha! The entire wi-fi network went down at precisely 3pm, funny that!
Monchote: On no!! Too many people streaming England’s game at work and the company’s network is about to go down.
PixelMagazine: everyeone in the office watching the match on their PC tends to slow the network down – give up calling tech support, it ain’t gonna happen
Mrmahoosive: If the BBC site buffers one more time I will take down our core network so only I can stream!
TvMiller: Americans turn to iPhone for understanding “offsides” call during World Cup match, AT&T network down leads to “bad call” row.

It’s no surprise that different matches caused problems in different places around the world depending on the teams playing at the time.  The countries reporting the highest difficulties usually had the following characteristics:

  • The country was actually in the tournament and playing at the time
  • The local rights-holder delivered the matches online
  • Football is a sport with a large following
  • The match was taking place during the typical working day
  • Typical bandwidth at work of less than 1Mbps per user
  • Online broadcasting is popular, promoted by the local TV broadcaster
  • Employers didn’t bring in large screens and encourage shared viewing

Various quality of broadcast were available, typically taking between 800Kbps and 3.5Mbps (HD) per screen.  Of course, in a business, this fights with normal business traffic, hence the negative impact on other applications and poor quality streaming for many users who did attempt to watch.  As most organisations do not have the means to minimize the effect of live video streams, such as with stream splitting, each new user accessing video adds another stream with the same network demands.  In large organisations, where Internet traffic is commonly backhauled to and from the data centre across the WAN, remote offices often suffered from poor Internet access and, worse, even to centralised internal applications.

It wasn’t just internal business networks that buckled under the weight; public wireless networks were under strain causing problems for travelling users.  Mobile data bandwidth usage in the USA increased by 24% and post-match mornings saw 32% incease in YouTube traffic, see this email from a colleague:

Due to recent bad weather I spent more time than I cared for at several large airports (Chicago, Washington DC).  In the past few days, the wireless service was bogged down by users watching the World Cup.  If anything exciting happened in the match, you could hear the shouts and fan reaction through the terminal as well.   Many times I had no signal, or very limited web access to email with virtually no ability to surf the web…the impact was real and noticeable.

Perhaps, for you, the World Cup didn’t impact your network.  However, it has lessons for us all – Are you ready for the next explosion in demand for popular content? These demands are happening with increasing frequency—take for example Tiger Woods press conference and President Obama’s State of the Union address in the USA for example. Local news or political announcements often create significant peaks too.  Some content can be predicted, but a global live newsflash can appear at any time.

IT managers need to look at their own statistics for the World Cup and plan for the next flood of content, look at each office and every country – watching the growing popularity of streaming to predict and put in place the solutions to ensure business traffic can be delivered during popular “stream-storms.”

There are a number of different approaches that IT managers can take. If you can predict the next set of content demands, some organisations may take a strict approach by attempting to block web access to all known sites that stream the content. For example, by using web filtering systems, IT management can block access to global sports sites, though users are likely to be unhappy and may still spend time attempting to circumvent the blocking. 

A second option would be to block the protocols used for streaming, however this may include all Real, Microsoft and Flash streams – and in doing so, block internal streams, streaming news and standard parts of web sites, interfering with work-related web information.  This approach will not work with on-demand video clips that are generally delivered as an integral part of the Internet flow.

Instead of either of the above approaches, organisations should look to adopt a more flexible attitude.   IT management can improve their network infrastructure to reduce stream usage through real-time stream-splitting, optimise streaming data or allow users to time-shift the content to be during normal breaks in the working day, as follows. 

Firstly, bandwidth management devices at the Internet egress point can be set to define one stream provider as “approved” and given a high priority (management then encourage employees to use that stream), other streams are lower priority or blocked. 

Secondly, appliances can be installed within the organisation’s network to split the streams – meaning that one external stream request can be sent to multiple users simultaneously.  This greatly reduces the upstream bandwidth required. 

Thirdly, WAN optimisation appliances that support stream splitting can be deployed between offices and at Internet connection points to take a single stream and divided as needed to serve user demand. 

Fourthly, many of the stream splitting appliances can also cache the streams, allowing users to time-shift and watch the game later.

Happily, this doesn’t mean installing four different kinds of network appliances, as there are some devices can deliver multiple benefits in one.

In this way, management can allow (or even encourage) video content whilst minimizing the load on the Internet gateway or branch office by caching locally through a proxy appliance and splitting a single video stream into as many are needed to meet the demands.

There are further benefits to installing solutions to optimise streaming content.  As streaming is embedded in so many business sites, the general load on the network will fall and quality of web content delivered improve.  Internal streams (such as CIO broadcasts) are optimised in the same manner as external streaming, and web video-conferencing between users and customers can be enhanced.