Are you browsing comfortably?

In the web development industry, wherever and whenever I’m talking to someone, and the phrase “Responsive Web Design” enters the conversation, I immediately think small: phones, tablets, and fitting the web into very small, very high resolution areas. Now that the Internet is encroaching on every aspect of our lives, this feels like an outdated notion, and that responsive, shouldn’t be a synonym for small width websites, it should mean responding to whatever is given to it. In this way, I investigated the state of the web on larger screens, and the largest screen I could find was my TV.

The TV is still the centre of entertainment

In the UK, There is a 98% chance that you have a TV. That TV might not be very big, but there is a growing trend for at least one large, over 32” screen, in the typical family house. Today, with all of our games consoles, set-top boxes and other peripherals, TVs are just a big screen, and one that is central to family lifestyle.

Our research shows that increasingly families are gathering in the living room to watch TV just as they were in the 1950s – but now delivered on bigger, wider and more sophisticated sets.
James Thickett, Ofcom’s Director of Research

Please don’t think that that’s an outdated statistic, according to the annual Ofcom Communications Market Report, we watch more hours of “TV Content” than ever, at an average of 4 hours a day. However, the definition of TV content has changed, and it now includes internet TV and video services through your TV. There are a lot of different ways to get these services, from your games consoles and set-top boxes, through to using a PC to drive the TV or having a Smart TV, where the TV can get these services itself. In fact, in the UK, 7 million households can get the internet on their TV (39% of the population)

Interestingly, the Smart TV market is growing at a quickening pace, but the total volumes of smart TVs are low, only 7% of households have a Smart TV. Therefore, it’s mostly peripherals that are causing this trend. Over half of all households have a games console, compared to 7% with a smart TV. According to Ofcom, there are more household with games consoles than there are with smartphones. This is a big market potential for the internet on the TV, as there will be plenty of other unreported access via new peripherals such as Chromecast, Apple TV, or through Media Centre PCs.

What’s the web like on a TV?

So, there is a market for the web on TVs, and it’s getting bigger as technology slowly creeps into every aspect of our lives. Looking forward, what in our current web stack can we use to make the web work on bigger screens.

For TVs, there are three things that are significantly different from regular user/browser interaction.

Where you’re sitting, what input you have, and what output the screen will give you. All of these apply to desktop and mobile browsing as well, but are very different for the TV, and must be taken into consideration. That’s a simplistic equation for designing experiences, and it works for this case, we just have to solve for all devices

Position

Now, when we normally talk about web design, we can safely assume that the user is around 6 to 18 inches away from the screen. Close enough for the vast majority of use cases, but with the TV, we’re about 6 feet away, and in the US, where people tend to have larger living rooms, they are around 10 feet away. Microsoft even branded this experience as the 10 foot experience and came out with a practical guide to developing for these user interfaces, and put it at the heart of the windows media centre experience.

This thinking follows a very basic principle: things look smaller the further away they are. In 3D graphics, if you want to make something look like it’s moving away from you, you make it smaller. Let’s look at a basic example of a smartphone…

On your smartphone, 12px of text (at a viewport height of 480) is about 40 lines of text in 3.5” is about 0.2cm for each line of text.
On the 480p viewport on a 3.5” screen, we get 40 lines of text, with a 1080p viewport, on a 42” screen (20.4” tall) means you can have 21 lines of text on the screen, instead of 40.

Therefore, for the 10” experience, you’ll need to almost double the font size.

This rule is good for the majority of well-sighted users, but our own vision ability varies significantly, whether you’re wearing glasses or not. As a default for the TV, you should double the text size, but you also need to let the user control the text size, and respond to that. We’ll come on to that later.

TV manufacturers have realised that text size is a big issue, and have changed how the browser behaves to correct this. This is usually done on the mobile web for us using the viewport and device-pixel-ratio. So, what’s the resolution of your TV? If you asked the person next to you, the chances are they’ll say 1080p, or 1920 x 1080 pixels.

Viewport size	Device	Aspect ratio
800 x 500	Wii	16:10
980 x 514	Wii U	19:10
960 x 540	Samsung TV	16:9
1024 x 574	Google TV	16:9
1041 x 586	Xbox 360	16:9
1094 x 928	PS3	11:10
1236 x 701	Xbox One	16:9
1280 x 720	LG TV	16:9
1540 x 754	PS4	16:9
1920 x 1080	Panasonic TV	16:9

Despite your TV being capable of 1920×1080, the browser will run at whatever-the-hell-it-wants, and sometimes at a completely obnoxious aspect ratio.

These here are just TVs and consoles. Think about the iPhone, and its viewport: 320×480, or the iPad at 1024×768. These are more like tablets than phones, though the interface we probably want to show is more akin to the phone than the tablet. Partly this is due to the 16:9 ratio (or something else stupid like the PS3)

The Web browser viewport did not provide a good guide to how things should lay out. In fact, relying on viewport alone lead to arguably poor designs.
Luke Wroblewski (@lukew) Nov 3rd 2013

We cannot make assumptions of working with TVs based on screen size alone. We cannot tell if the user is on their sofa, or on their laptop resizing their screen.

Input

When I think about telling my TV what to do, I think of my remote control. Now, with normal TVs, Sky boxes, DVD players, audio systems, they all come with their own remote. When I think of interacting with a web browser, I think of a keyboard and mouse. These two methods of input are practically polar opposites, so, over the years, companies have come up with a lot of different ways to control their new Smart interfaces. Full keyboards, mini keyboards, mini touch screens, magic wands, they’ve tried everything, and still the most common mechanism is the D-pad on the remote control.

Nearly every Smart TV also has a companion app to help improve the user experience, providing a large, context-sensitive input pad, with a decent keyboard improves the input experience into something that might actually be worthwhile. I especially found this whilst testing on my Xbox 360, the SmartGlass app, combined with the xbox controller felt quite good, except that I still had to use two controllers to work a browser, so it’s not ideal. Still, it’s the best there is right now. Voice control through the Kinect promised so much on the Xbox 360, but it didn’t appear. On the Xbox One, and on the PS4, interaction is more fully-featured, but it’s still little more than the basics of navigating a website.

All that said, voice input is another mechanism that we, as web designers, need to consider, because this technology is just around the corner, and it complements all of the other inputs that we have and those I’ve just described

So, the d-pad is the only input you can rely on, but even that is an assumption that fits most categories, for example, the Xbox 360 doesn’t have a D-pad, but does have a cursor that works quite well. All other forms of input that are available feel like a bonus, but actually, it’s probably more of a problem. The more inputs there are, the more inputs you have to design for and the more situations that you have to take into account. The d-pad, which maps to the arrow keys, is relatively simple to code for. Creating an interface that works well with the d-pad, and a cursor, and a touch pad, and potentially voice, is a nightmare. Add in a custom wand-style motion API and you’ll end up designing an interface that doesn’t suit any of these input types, creating more problems for yourself than you solved.

Getting input right is a difficult problem, and right now, there’s not a lot that you can do. There are some efforts going right now, with Patrick Lauke working with the Touch and Pointer Events working group to add d-pad events into the pointer events specification, hopefully unifying the different events specifications and making it easier for developers to implement. There is also a proposal for d-pad events in the W3C DOM Events level 3 spec, which is currently a working draft

Output

When I presented this at the Side View conference, I had asked my twitter followers to fill out a survey on Smart TVs, what they had, how they used them, and what their experiences of the web on TVs was so far. Universally, the responses were negative. Here are some answers to the question do you browse the internet on your TV?

No. Experience is awful.
No because the TV processor is really slow.
Rarely. It’s easier to use phone/tablet/laptop which are usually all in the same room as the TV.
No. I find the entire process annoying and tedious.

My own survey for this conference revealed that there are lots of you who have internet-enabled TVs, and that use them for internet content – but it’s almost universally a bitch to do and simply not worth it compared to gathering around a tablet.

From the comments, I noticed that the UI appears to be really slow, and that really hurts the experience of using the TV. I investigated what the specs of your typical Smart TV are and was shocked to find that first-generation Smart TVs have a processor equivalent to an original iPhone. Even 2013 models are using 1.2GHz Cortex A9 processors – which is approximately iPhone 4 standard. In the end, it’s a lot easier to pull out the smartphone in your pocket, that has the same resolution, and with websites and apps that are designed for it, not to mention it’s more powerful, and has more RAM.

Can they all really be that bad? I went looking for a browser benchmark and came across HTML5 test results for a number of TVs and consoles

TV browsers themselves aren’t the best for web standards, especially looking back a few years. The Wii and PS3, some of the first mainstream browsers on TVs, score sub-100 HTML5 test scores, and even IE9 on the Xbox 360 only scores 120 when it was introduced two years ago. Compare that to the iPhone 4 that was released the year before, almost doubling that score. Smart TVs themselves run a variety of browsers, usually Opera (Presto engine) or NetFront (webkit). Older versions of NetFront browsers don’t come anywhere close to passing recent web standards tests, with NetFront 3.5 scoring 11/100 on the Acid3 test; sadly the 2012 models of LG’s Smart TVs still shipped with this browser.

Thankfully, since NetFront 4.0 on the PS3 and the NX version on the Wii U and PS4, it has been passing the Acid3 test. Looking at the latest results of the HTML5 test, some Smart TVs are now surpassing the 400 mark; even the PS4 and Xbox One don’t get that high a score. So, the future looks better for web standards support on TVs, but the chance of you having one of these devices is slim: some TV sets now have a viewing lifespan of 60,000 hours. At 8 hours a day, that’s 20 years before you replace the TV.

Get to the point, will it ever work?

It looks a pretty sorry state doesn’t it. The overriding question for me is, “Is it even possible? Can you make a responsive UI for all sizes and situations?”

I looked at YouTube, and BBC iPlayer for their desktop and mobile efforts. The first thing I noticed that whilst their desktop sites are responsive, go to the website with an iPhone or iPad user agent, and you get redirected to m.youtube – however, when you visit as a TV, you get a completely different interface, something that YouTube calls leanback mode. It’s a crossbar style interface that you interact with mouse or keyboard arrows or a d-pad. Compare that to the higher resolution UI on the desktop and I start to think that responsive design doesn’t work on TVs

BBC’s iPlayer behaves very similarly. iPlayer’s Bigscreen mode is a keyboard-accessible website, with a very different HTML structure to the desktop website, though it’s far closer to the look and feel of the regular iPlayer website than YouTube’s leanback mode is to its desktop website.

These two are the big boys in the playground, with services designed specifically for the TV and with the budget to design for this screen size and capabilities in particular. I wanted to see how more of the web works on a TV.

Using IE9 on the Xbox 360, I tried the BBC News website. The Xbox’s viewport is 1041*586px, which should trigger the responsive BBC News, except that , it UA sniffs to provide the right website, and the main page isn’t optimised for large screens, giving padding around the edges (inserted by the Xbox) Fonts aren’t big enough to read comfortably, but you can still just about read it from 10 feet away, but it’s not easy, and if you’ve not got 20:20 vision, it’s probably very difficult to read comfortably.

The point is that for the TV, you have to change your perspective on the design, drive content directly and simplify.

Targeting the TV

screen size doesn’t give us a complete picture of what we need to know to design an appropriate interface [for TV]”
Luke Wroblewski Oct 31st 2013

TVs have very similar feature sets to desktop browsers, similar resolutions, and viewports, the very things that responsive web design has come to rely upon. There has been a CSS media type of TV for many years (CSS2 Media Types), but support for it is non-existent because screen types are mutually exclusive. When a browser is created, the manufacturer picks a media type for it to respond to, and only one. Because screen is by far the most common media type, that is the one that all TVs have picked. So, the only way to detect a TV, right now, is user-agent sniffing. A technique that the web community has admonished for years is now all we have.

There should be a better way, and if we can’t tell that it’s a TV, perhaps we should ask the user to tell us.

Trying to tackle this problem head on, Aaron Gustafson of Adaptive Web Design fame, is developing a plugin that will give you a button that lets you enter couch mode, simply increasing the size of the font using the width of the viewport as a base. Not all Smart TVs support these units, but all the current consoles and modern hardware do, so give it a try to provide your users with a simple option to turn it on and off.

We can also use techniques to assist users who increase the font size in their browsers a lot, which many users of TVs do because they’re far away from the screen. The simple way to do this is to define media queries in ems. When you increase the font size by zooming in on a browser, this also increases the base font size. Therefore, if you zoom in with ems based media queries, the number of ems it takes for the width to fit the screen decreases.

This is in contrast to a pixel view, where you can’t increase or decrease the number of pixels on font size, you instead change the width of the window. Now, trying this on Chrome on the desktop will give you a mixed result, as the zoom function actually reduces the size of the viewport and re-fits the page to match. However, older browsers, such as those on TVs, simply increase the base font size, making ems the only way to go for responsive widths on large-text screens.

The future is brighter

All of this sounds very doom and gloom on the “designing for TV” front, with traditional responsive design not being able to cope with the complexities of a distant user perspective. However, the future is brighter for TV browsers at least. The next generation of consoles achieve decent scores on the HTML5 and CSS3 Tests (PS4 Xbox One). You can now get a very good standards-compliant browser on your TV, and there are a lot of ways to do it, through set-top boxes, Airplay, Miracast, games consoles and the TVs themselves. Many of the reviews of the Xbox One have complimented the speed of the dashboard, enabling a superior user experience and potentially, making web browsing on a TV a reality.

There are other innovations that are disrupting the living room market, and Chromecast is one of them. It’s the internet TV experience bottled into a $35 package. Many household won’t be able to afford the luxuries of a £429 Xbox, buying a new TV, or building a new PC or Mac to hook into their TV, but you can probably skip a family meal out and buy two of these incredible devices.

Looking back to look forward

Six years ago, when the iPhone brought about a new age for mobile computing, we reacted by creating responsive web design, and adaptive web design to allow our content to be suitable for devices of varying screen size, but these techniques fall flat when the screen moves from the position that we think it should be, about a foot from our face. TVs are the tip of this iceberg because they are the easiest thing to adapt to our mechanisms, after all, they will accept any input that we give them through their plethora of ports.

The wearable technology trend is another threat to our current mindset, with the Galaxy Gear running full Android, it’s not unforeseeable that bigger screen devices will emerge with curved screens, foldable screens, dual-screens, second screens and companion apps. And these are all the ideas we’ve already thought of – who knows what will be around the corner

We can no longer assume based on the size of a screen, what someone is doing and how they may want to use a website. Ethan Marcotte recently put this rather eloquently…

The number of problems we can solve automatically for our users are dwindling.We can’t know reliably how much bandwidth a user might have available to them, whether they’re outside or stationary, or whether they’re mirroring their display to a wider screen—or, or, or.
Ethan Marcotte Nov 2nd 2013

Ethan goes on to say, “What we can do is solve the problems that we can solve and acknowledge the gaps, invite our users into these problems, and let us help them provide them with the ultimate experience for the app”

We can solve these situations, but we have to ask the user to give us the context that they are using their app in, couch mode, glass mode, watch mode, the list goes on.

One day, we will work out how to provide that seemless experience to everyone in every situation, but until then, we can only work with what we’ve got. And so, if you’re designing for those situations:

Solve the problems that you can solve.
Ask for help when you can’t.
We’ll solve the bigger problem together.