A personal example of Big Data crunching

I saw Stephen Wofram's blog entry where he published analytics of his life for the last 33 years of emails, telephone calls, calendar entries etc.

http://blog.stephenwolfram.com/2012/03/the-personal-analytics-of-my-life/

At first, some people may wonder whether this has any benefit and what the data analytics are for, but I think it shows firstly the sort of data that can be graphed and (having always loved graphics more than text myself) the greater benefits are from seeing visually any long-term trends and allow the individual to decide whether to change some things that they do (emails on a Friday night, perhaps).

For the rest of us, it has information too.  For example, there have been discussions on when to email or tweet for maximum impact, with a large dataset we could see when users tend to be at email already and also when other emails aren't being sent, perhaps both can help show the most productive times.

We can see when meetings are set - knowing a target's norms allows you to fit in with them.

Changes in behaviour over time can also be shown - personally I'd love to know what percentage of emails I bother to open, what number I read on a mobile device and whether I do or don't download the images - I am sure the percentage of fully-read emails has reduced over time.

I think in the workplace one very useful piece of data could be the ratio between meetings and "non-meetings", I wonder sometimes how some people manage to achieve anything at all if they have 6 back-to-back meetings each day, as there's so little time to actually perform the actions agreed.

So, though reading his blog may at first make you wonder whether it is useful, I think its a great indication of what can be gleaned and if we multiply that data by every individual in an organisation, it can show the best time for internal meetings, the best way of communicating, the types and methods of communication being used - first get the data, then analyse it, then look for patterns and make the difficult jump between facts, data and information.

Of course, he has the benefit of using the same systems for many years - for most of us bouncing between jobs and various email addresses, phones etc. we probably don't have the data itself.  So, step one is to make sure the data is being tracked and archived, even if we can't work out how to extract the value today, that may come in months of year's in the future.

Someone once said, the best way to find a needle in a haystack is to remove all the hay and what you are left with is the needle.  Step two is then crunching the numbers and looking for the patterns that are useful.

Call me a geek, but I think its rather fascinating.

Wi-Fi Nearly Everywhere

The incumbent national operator always comes in for stick, it doesn't matter which country you go to, the poor old national carrier is usually seen as more expensive, slower to roll out new features and therefore less innovative....

Here in the UK, it is no different and poor old BT have been criticised for decades, despite having to provide a universal service across the country, run uneconomic pay-phones, keep different businesses of their network separate and provide the backbone for their competitors while constantly negotiating with the regulator.  (Ignoring that they have been kept out of the mobile market altogether to ensure that they didn't squash the nascent mobile market all those years ago).

They will be happy, I hope, that this blog isn't to knock BT, as I want to praise one thing that they have been rolling out for a few years that may have passed you by, at least until it grew to its current size and they started promoting it on their latest TV adverts ... Their huge installation of Wi-Fi networks that other BT customers can share.

This advert says that that they have 3million Wi-Fi hotspots and these are each delivered by their own customers - though I notice the note below the advert now claims 3.5million.  Whoever thought of this was a genius, at the time they started, Wi-Fi wasn't as ubiquitous as now, but as each of us gather more Wi-Fi enabled devices we all want Wi-Fi access wherever we are and as more location-specific applications are launched, we get more addicted to being always on.  So, congratulations Mr or Ms BT.

http://www.youtube.com/user/BTCampaigns?feature=pvchclk#p/u/0/2sWsKpcUKE0

So, I thought I'd do a bit of maths...

A typical Wi-Fi radius coverage is claimed to be between 46M (indoors) and 92M (outdoors), so if we assume a radius of 75M, then each area of coverage could be around 1,7671squareM.  that is around one fiftieth of a square km

The UK's area is 243,610 square km - so 12 million hotspots all placed equidience apart would cover the whole country (countries), so if only the 3.5 million spots they have now were spread around, BT customers would have free Wi-Fi access in a quarter of the country.

Of course, its not like this - if you look around in built-up areas you'll often find multiple hotspots, but still its pretty impressive and with the local areas putting up free services (Westminster for example) and free services in restaurants and hotels - the coverage model is constantly growing.

Now, if only the devices would connect without any user intervention, default to Wi-Fi before 3G and 2G and the confusion in my head about the different between BTFON, BT-Openzone and BT-Openzone-H was cleared up, we'd all be able to have Wi-Fi "nearly-always-on".

I have an iPad that is Wi-Fi only and it has been a very rare occurrence that I've wanted access outside a hotspot and thought I wish I'd bought the 3G version - and each day as another bunch of hotspots gets installed, that will get less and less frequent.

Will there come a day when many of us use mobile phones that are Wi-Fi only? Just designed for data services, the user then uses Skype or equivalent for phone/video calls and the need for 2G, 3G and even 4G services goes away altogether? I think so, and I think it is closer than some people may think.

Here's one from Samsung, I wonder how many they sell and the growth that they are seeing -

Galaxy S

Hotel Wi-Fi - Great Differentiated Service

Having just come back from a week's holiday, I'd like to praise the Wi-Fi service offered by the hotel and wish and hope that others would do the same.

Firstly, the hotel is situated in around 12 hectares (20 acres, I think) of land with buildings scattered around, so I can imagine that coverage is tricky, but though they didn't guarantee coverage in every room, it worked everywhere I needed it.  I could see a range of external access points and they had the standard Wi-Fi logo on them, so anyone without coverage would know where to walk to get closer to an AP.  Not sure which vendor provided the APs, though I see that Ruckus Wireless seems to have a huge amount of the hospitality market with their Smart Wi-Fi providing greater distance and steering around obstacles, so perhaps it is them.  www.ruckuswireless.com

Then, each AP was advertising two different networks - a free one and a paid-for option.

The free network had limited access; no VPNs allowed and content filtering with various forms of adult content blocked, no streaming content from YouTube or even streaming content on other web sites, no downloading of applications from various app stores, shareware sites blocked etc. and a statement that it was a best efforts service, so presumably some QoS service that gave the paid-for service higher priority.

The paid-for service had a flat-fee of around $50USD a week, then you get open access, can use VPNs, watch streams and QoS priority.

I think this is great mix, it allows the customer to choose the service that they want and whether to pay for it.  Even the free service allowed me to catch up on news, Twitter, pick up my email and I thought to myself that I can wait for the new version of an app. and download it back at home.

All in all, I wish every hotel/airport/station would follow the same path.

And as I was praising their service, I'll give them a little plug - food and service was great too so if you are ever in Mauritius....
http://sugarbeachresort.com/

Google - just a feeling in my bones

I have two blogs - this one and another on a completely different subject.

Now this one is "well Googled" - if you use Google to search for it or words and phrases in it, it comes up as you'd hope it would..... but my other blog has taken a long time for Google to find (though Yahoo! and Bing have it indexed and you can find it there easily)...

Now, before I accuse anyone of anything, of course, all the search engines go around indexing the web at different rates and one anecdote does not make a rule, but this blog is on Blogger, which as you probably know is owned by Google and my other blog .... isn't.

Hum.... just wondering

UK 4G - Too little, too late?

Ofcom has announced new plans to auction the 4G mobile phone spectrum, but will it deliver the windfall the government is hoping for, will operators invest in the coverage expected, will customers actually use it and will the operators therefore make their money back?

Well, I don't know.

You can read about the Ofcom announcement here http://www.guardian.co.uk/technology/2012/jan/12/ofcom-outlines-4g-mobile-spectrum-plans but let's look at it from the users' point of view.

I own a phone, iPad and computer - all able to use the Internet.  They are generating and downloading lots of data, all day whether I ask them to or not, sometimes you just try stopping them!  There's no doubt that the amount of data I and my family are transmitting and consuming is growing, not least as emails are being sent to all three devices simultaneously as they all try to keep synchronised.  I've written before about the automatic downloads of new TV episodes from iPlayer, there's surely no doubt about the growth in traffic.

We also as consumers want it to be ever faster, higher performance, no waiting, HD-quality and whatever is the best way next to consume content; IMAX or 360 degrees with multiple projectors coming out of your Mac in a few years?  Who knows?

But, do we need 4G?

Today's devices can connect to a multitude of data sources, of course this being communications we have a huge bunch of acronyms to confuse the enemy - GSM, 2G, 3G, Edge, CDMA, UTMS, WiMAX and 4G, and the devices are smart enough to pick and choose the best available system and switch between them without bothering the user.  Maybe they don't always make the best decisions, but as the algorithms get smarter and they can take into account maximum throughput, reliability, signal strength, achieved throughput and continue to keep connected while making incredible leaps between systems, we users can be oblivious to the technology underpinning the data.

So, what is in it for the operators?  Why should they invest in 4G?  Well, to give the fastest connection to their users and steal customers from the other operators, as almost everyone has a connection today, they have to compete on either performance or price (or a bit of both).  Sadly, the up-front investment costs could be huge, as some 4G signals don't travel as far, they may need more aerials or have greater "not-spots" for 4G than 3G.

OK, but how much will we pay?  I am currently on a PAYG deal that costs me £10.00 (USD$15) a month, I get unlimited texts, unlimited data and 250 minutes of UK calls.  (If you'd like the same deal, just let me know).  Prices are coming down, so can the operators make any money?

But it gets worse for the operators, what do I connect to the most?  Actually none of the above acronyms - I use Wi-Fi.  I find Wi-Fi spots in so many places today and a huge number are free or available in a package with a single provider (BT's deal where you can share other consumer's Wi-Fi for example) or offered by cafes, pubs etc. - even local councils are starting to cover the area at limited or no charge.  You can download free applications for your phone or iPad that lists over half a million wifi spots globally and simply walk to the nearest one.

I can see 4G becoming a little like satellite phones from years ago.  I remember a few folks with them telling me how great they are "as I can make calls from the top of Everest ... anywhere, no patchy connections".  In those days, there were a huge number of areas where cellphones didn't work at all, but I always wondered when I would actually be at the top of Everest and, as each new cellphone antenna was installed, the need for the satellite phone reduced.

So, I welcome 4G, anything that improves connectivity is great, but the operators are smart (hence the legal challenges over the last couple of years) and they can see the consumer pricing going down, so where's the ROI for them?  But the big one for me is that as each new wi-fi system is installed, the need for 4G technology will reduce and unless the same operators are getting revenue for the wi-fi connections, where will the money come from to make the investment that the government wants them to sign up to?  (Actually, as an aside, perhaps they will get MORE revenue from rural areas and not less, as cities covered in Wi-Fi might mean less 4G revenue).

There's a lot of other aspects to take into consideration, not least coverage distance, but essentially it was coverage that was the great selling point for satellite phones - now used in very remote places around the world but you wouldn't expect to see one in most business meetings, personally I doubt I have seen one for 5 years or so.

So, there better not be any more delays, the height of the price that the operators will pay has possibly already passed.  The UK treasury may be hoping for a £3B ($5B) windfall, and perhaps they will get it as the operators don't want to have no 4G service, but on the other hand.... maybe not.  We just have to wait and see.

Ten Mistakes in Web Security

Web-based threats are constantly changing, yet I see people who's installations and policies have stayed the same for years, ageing and providing poor overall defences.  So, the beginning of the year seems a good opportunity to review the mistakes I see time and again, let's hope you read this and say that you aren't suffering from any of them.

1. I have desktop anti-virus, that is enough.  Well, hopefully enough information has been shared to remove this misunderstanding.  Employees can be fooled by phishing attacks, can inadvertently send out confidential information and sadly the constantly-chaning nature of malware means that anti-malware systems are constantly playing cat-and-mouse to try to keep up with changes from the bad guys.

2. Web filtering if for blocking only.  Web categorisation systems can block, but can also report, coach users, IT can set priorities based on categories, users can be redirected to other sites and can selectively identify and control particular aspects within a category (such as allowing reading but no downloading of executables from untrusted sites).

3. I only need one simple policy.   The "One size fits all" advocates can often set a simple policy (Eg. block adult content, phishing and known malware) and then leave it for years.  Almost all organisations have different users who need different access and policies should be reviewed regularly to make sure that they are up to date.  Suppliers tend to add new categories and enhancements each year, make a note now to review your policies every few months.  (Such as block all older browsers that have known vulnerabilities - go on, do it now).  With thousands of application within Facebook, some for business and many not - you can define policies that see inside social networking and give you the control you need.

4. Reputation systems are enough.  Security systems based on reputation are good, for email!  Looking up an email sender is a great way to start reviewing whether an email is likely to be spam or infected.  However, web pages are so dynamic that reputation systems can only offer one of many pieces of information needed to review content.  In tests, those systems based on reputation have performed poorly on real-life web threats.

5. I don't need to warn/train my users.  Making a new employee sign an Internet Acceptable Use Policy when they have many other concerns on their mind is hardly going to change behaviour and even irregular update courses don't achieve much.  But make sure that users receive splash pages when they access the internet and warning pages when they attempt to access a page with restrictions and they have a constant reminder of the organisation's policies.

6. My organisation isn't a target.  Every organisation has confidential information in it, every employee can be a target even if just for their own knowledge or personal data.  We are all targets.

7.  I cannot control remote users, so I won't try.  With almost every user taking company information outside the offices on tablets, phones or laptops - every organisation has to work out what to do for their mobile workforce.  Encrypt devices, sure, but also look at the policies that can be implemented on mobile devices, make this year the year you investigate all the mobile security options, you know you want to... :-)

8.  There's too much in the logs, so we just keep them in case anyone asks.  Your logs can be very useful information, showing users behaving oddly (how many hundreds of MB downloaded from your cloud-based CRM system?) the most popular categories of the web, the loading at particular times of day or week and this can help you plan new policies, advise users and design a better, stronger network.  Just one example; logs can show which PCs are already infected and a policy implemented to take them off the network and the user informed when they run the browser "Go and get your PC cleaned".

9.  IT define all the policies on their own. Appropriate policies should be designed together with senior management, employee or user advocates, legal and HR departments together.  Get together and ask each other "what-if" questions, the flexibility available now to define policies by group, by office, by time, different policies based on applications within web sites means you really can target appropriate policies.

10.  New applications and systems make it impossible to keep up.  You can't just throw up your hands and say that you can't control Facebook, Twitter, LinkedIn, Foursquare, Skype and all the other newer social networking applications - there are many different controls that can be implemented.

Web control is all about balance - allowing users to access those areas that they need to, but keeping them safe online from those areas they shouldn't.  But the security target is constantly moving, keep yourselves up to date and your users will be safe, your data will be secure and employees kept productive.

Anyone can reuse this top ten list as long as I am quoted as the source.

Content rights in 2012 - time for a bold move online?

I have written about the move to from TV to online content for many years - with time-critical content (sports & news especially) being watched more online as each year passes.  See previous articles on the 2010 World Cup, the BBC iPlayer statistics, LONAP graphs etc.

The content-owners obviously want to maximise their revenue and in the past this was best done by splitting rights by country - if you own the rights and can sell it over 200 times, you can negotiate your best deal in each individual market.

When a broadcaster relies on subscriptions for its revenue (such as Sky in the UK), they need to keep buying the most popular content to keep their subscriber base with them and the content owners have managed to use this to keep the revenue flowing.

So, the major sports have continued to grow, reinforced by a circle of more PR leading to more press, higher salaries for the sports stars, higher rights payments and the money keeps flowing around and everyone is seemingly happy, even the consumers keep paying, though perhaps with grumbles.

Meanwhile, non-subscription-based broadcasters find it more and more difficult to compete, either dropping sports altogether or slowly drifting down the scale (a bizarre manifestation of this being the shared rights between the BBC and Sky for this year's F1, my guess is next time around there BBC will drop out altogether).

I believe that the current model is unsustainable and the balance of power will shift in part by technology and partly due to multi-country deals.

Firstly, the European Court of Justice has ruled that any EU citizen can get their TV service from any provider - see the case brought by Portsmouth landlady Karen Murphy.  If content-owners continue to negotiate country-by-country, then the cheapest buyer/broadcaster can undercut others to broadcast its content outside its normal area, bringing down all prices - the content-owners (and other broadcasters who have paid a higher price) aren't going to want that, so they are likely to try to negotiate wider rights - perhaps the whole of the EU or even a global deal, if a single or group of broadcasters can afford it.

From the technology-side, it is ever easier for consumers to jump around the web (spoofing their IP addresses, using proxies in other countries etc.) and find the content wherever they want it, as long as someone is broadcasting the content on the web somewhere, it can be found and watched or listened to.  As this is the case, why should a broadcaster in one country keep paying an ever higher price at each renegotiation when some of their customers might be siphoned off to someone showing it on the web in another country?

Meanwhile, YouTube and other broadcasters are looking around for new ways to bring in eyeballs.  This article brings in Google/YouTube and al-Jazeera as possible bidders for the English Premier League with Apple (perhaps) on the sidelines.
http://www.guardian.co.uk/football/2012/jan/04/apple-premier-league-tv-rights

So, what will happen in the future.  At risk of making a prediction that has many vested interests fighting against, I think that we may be at or near the top of the right-holders price-curve.  I believe that a mixture of technology, legal rulings, economic concerns and ever-greater leakage of content onto other platforms will reduce the overall prices for content owners.  It will be interesting to see what happens when a major sporting event moves to a non-traditional broadcaster - maybe 2012 will take us further down this road.