blue       earthtones
Streaming Video with HTML5

Discussion: I've had the requirement many times over the course of the last four years to post-produce and publish a wide range of video on client websites. During this time, technology has taken some great strides forward which simplify the task and improve the results. The solutions I've employed began historically with Flowplayer, and have in more recent years shifted to the HTML5 <video> element. But given the persistence of Internet Explorer 8 in the global browser population, both solutions still prove to be highly useful, albeit with the HTML5 <video> element as a preferred method, and Flowplayer the able fallback with .mp4 for IE 7-8.

That being said, note that although this page was built with the (formerly free unbranded version) of Flowplayer as the fallback player circa 2009, I've switched to more recent alternatives that emerged in the time since then. In particular, I've become fond of and have extensively used the open source JarisFLVPlayer (available on SourceForge) since then, including as recently as January 2014, to provide video previews for a major Video on Demand provider. See more regarding video players in the Video Players accordion tab below.

Solution: Bear in mind that it's still imperative to factor in the visitor's browser capability, particularly including mobile presentation, when implementing and testing video. Due to the absence of the Flash player on Apple mobile devices, I've ceased producing .flv versions of my clients' videos in favor of .mp4. In view of such factors, it's necessary to code for, test, and remediate a browser set including IE 7-8-9, Firefox, Chrome, Safari, and mobile Safari (iPad/iPhone) as a minimum. Experience has taught me to visually QA with at least nine (total) browsers: Windows (6), MacOS (2), and iOS (1). Don't neglect to factor that requirement into your consulting quotes or Agile project sprints.

That being said, HTML5 <video> is refreshingly predictable (read: reliable) in current versions of the A-Grade browsers. Nevertheless, cross-browser consistency continues to present a testing exercise to web developers, mostly related to which video format(s) ('containers') a given browser will support, and the width and height specifications of both the <video> element and the <section> or other element you display them in. For example. specifying just the video element height in css and allowing the browser to calculate the width will result in different (and unsatisfactory) results in Mobile Safari as compared to Firefox or Chrome.

Video formats, aspect ratios, containers, conversions, codecs, and other theory are a fascinating subject which is both broad and deep. However the objective of this article is to help you get quickly started using HTML5 <video>, so I'll focus here on just those details you need to understand, and forego many of the peripheral technical details.

EXAMPLES OF HTML5 VIDEO

The Examples:

  1. Example 1 shows a video converted from an anamorphic 720 x 480 (1.5:1) .vob file taken from NTSC DVD, and transcoded as a 400 x 300 (4:3) source in three formats (webm, ogv, and mp4), then presented below at 320 x 240 px.
  2. Examples 2 and 3 present videos converted from high-definition .mxf (clip) files taken from P2 cards in a professional video camera (Panasonic AG-HPX) shooting DVCPRO HD (16:9). They're transcoded as 320 x 180 (16:9) sources in the same three formats as Example 1.

Don't let these video origins throw you since the only differences between these and a .mov video shot with a consumer-grade camera such as the Nikon or Cannon in your pocket, or a mobile device such as your iPhone, are the steps for producing the video <source> files. The rest of it — that is, the web presentation with HTML5 — is identical for all using the the code shown in the "The HTML5 CODE" tab below.


HTML5 Example 1

I clipped this video for sscrf.org from their DVD using Handbrake to load the .vob file and convert it to .mp4, edited this scene with Final Cut, and encoded the alternate .webm and .ogv <sources> with Firefogg (see the Software Tools tab below). For visual comparison here, the resulting video is presented at right at the same 320 pixel width as Examples 2-3, but in a 4:3 (or 1.33:1) aspect ratio rather than 16:9.

File sizes for this 41-second video:
  • original anamorphic .mov @ 720 x 480 px = 11.147 MB
  • converted .mp4 @ 400 x 300px = 3.176 MB
  • converted .ogg / .ogv @ 400 x 300px = 2.161 MB
  • converted .webm @ 400 x 300px = 1.561 MB

Your browser () is playing the .



HTML5 Examples 2 and 3

Here's an example of two short cuts of high-definition, widescreen (i.e. 1.78:1 aspect ratio) video that I imported from a pair of 32GB P-2 cards taken from a professional camera (Panasonic AG-HPX) shooting DVCPRO HD in January 2010. I imported the clip files with Final Cut Pro (under MacOS Snow Leopard), roughly post-produced the raw video with Avid, compressed and encoded it with Sorenson Squeeze, then more recently added the .webm and .ogv files using Firefogg. These are presented at 320 x 180 pixels (16:9 or 1.78:1), that is, "widescreen". Note the difference in aspect ratio (height) between these (16:9) and Example 1 (4:3).



BASICS

Here are some "first principles" to understand when you're designing for video presentation:

  • Aspect Ratio: Aspect Ratio is simply the width of the video source divided by its height. Standard TV video (and most 'consumer' cameras) reflect an aspect ratio of 4:3 or 1.33:1. This translates, for example, to 400 pixels wide by 300 pixels high. Modern "widescreen" (and most preofessional video cameras) produce video at an aspect ratio of 16:9 or 1.78:1. The latter translates, for example, to 854 pixels wide by 480 pixels high. (There are also some HD and super-wide formats not discussed here.)

    That being said, if you are planning to take your video source from DVD, you will find that it's typically stored in a .vob file with an aspect ration of 1.50:1, commonly 720 pixels wide by 480 pixels high (DV). This is because it is stored 'anamorphically' so that a DVD player can morph it to either ratio by manipulating PAR (pixel aspect ratio).

    For all practical purposes, the upshot of this in context of preparing DVD video for presentation with HTML5 is that in order to avoid distortion and loss of fidelity you'll need to morph your DVD source to the aspect ratio you plan to use on your web page. Do this before converting the result to webm, ogg, and H.264 formats for your <video> sources. (more at Software Tools, below)

  • Video Formats: With regard to video formats that are most useful for web presentation with HTML5, perhaps the most common, and certainly those that you'll need to prepare for/provide, include: ogg (and ogv), webm, and H.264 (mp4, mpv). Multiple formats (that is, <video> <source> elements) are required since the A-grade browsers play a non-orthogonal set of formats. Furthermore a fallback is required for IE 7-8 which do not implement HTML5 <video> functionality.

    The mime types for the encodings you'll require are:

    • webm "video/webm"
    • ogg/ogv "video/ogg"
    • mp4 "video/mp4" (H.264 in an MP4 container)

HTML5  <video> <source> BROWSER SUPPORT

These are current as of this writing (Feb 2012) but video container support is still evolving, so I'd also suggest visiting Wikipedia and other sources for the most current and more specific version details.

  IE 7-8 IE 9 Firefox 10 Opera 10.6 Chrome 17 Safari 5
Trident Trident Gecko Presto WebKit WebKit
webm No No Yes Yes Yes No
ogg No No Yes Yes Yes No
mp4 (H.264) No Yes No No Note Yes

Note: Google announced plans in Jan 2011 to remove H.264 support from Chrome "in the future".

Given the HTML5 code:

<video id="example" controls="controls"> 
	<source src="video/example.webm" type="video/webm">
	<source src="video/example.ogg" type="video/ogg">
	<source src="video/example.mp4" type="video/mp4">
</video>

    (plus the flowplayer fallback code in the "HTML5 CODE" tab below)
	

then, the following were observed as of the date of this writing (Feb 2012)

  • Firefox plays .webm (v 10.0.2 - gecko) - earleir versions play .ogg (not .ogv)
  • Chrome plays .webm (v15.0 - webkit)
  • Safari plays .mp4 (v5.1.1 - webkit)
  • Mobile Safari plays .mp4 (iPad/iPhone)
  • Opera plays .webm (v 10.63 - presto)
  • IE 9 plays .mp4 (v9.0.4 - Trident 5.0)
  • IE 7-8 play .mp4 with Flowplayer (the <video> fallback)

SOFTWARE TOOLS

After foraging around the web for over a year, downloading and testing a wide range of open source and commercial products for encoding video, I found that the following set of reliable tools best met my video editing and encoding requirements:

Free:

  • Handbrake is an excellent open-source utility which is available for both MacOS and Windows. This is an indispensible tool for taking video from an anamorphic DVD and converting it to .mp4 in your desired aspect ratio, cropping, re-sizing, plus a wide range of other useful conversion options. $free http://handbrake.fr/downloads.php
  • Firefogg, which is a Firefox extension, is an excellent free tool for creating both the.webm and .ogg video files required for the <source> elements in your HTML5 <video>. $free http://firefogg.org/

Commercial Products:

  • QuickTime Pro (Apple) is a highly-affordable ($29) upgrade to QuickTime that provides the ability to perform minor edits to mp4 and .mov files. Available for MacOS and WIndows.
  • Final Cut Pro (Apple) is an affordable post-production tool, and indispensible if you're taking .mxf 'clip' files from a professional camera, editing the clip(s), adding effects, and converting to QuickTime .mov. MacOS only. $299 http://www.apple.com/finalcutpro/
  • Sorenson Squeeze is a sophisticated video encoding and workflow tool. This professional product is enormously flexible for converting, encoding, and compressing video and audio. If you will be working with a large number of videos and formats on a regular basis, it's well worth investing the $599 price. http://www.sorensonmedia.com/

TRANSCODING THE REQUISITE HTML5 VIDEO FILES

As you may have noted from the introduction to this article, video originates in a wide range of formats and containers. But for both simplicity and all practical purposes, just consider that you'll most often be starting with a QuickTime movie (.mov) in hand. To emit the videos for these examples, it was only necessary to create compressed versions from the .mov source in three formats.

  1. .webm  for the canonical <source>
  2. .ogv  for the first alternate <source>
  3. .mp4  for the second alternate <source>, and Flowplayer fallback

I generate and specify <source>s in this order in deference to the smallest file size, and consequently, the lowest bandwidth burden on my server. The visitor's browser will determine, in order, which format it will choose. For example, here are the various files sizes for the 41-second movie used in Example 1:

  • original anamorphic .mov @ 720 x 480 px = 11.147 MB
  • converted .mp4 @ 400 x 300px = 3.176 MB
  • converted .ogg / .ogv @ 400 x 300px = 2.161 MB
  • converted .webm @ 400 x 300px = 1.561 MB

Foregoing (or following) editing and effects, the simplest method of converting a source .mov to the required HTML5 video formats is a straightforward process using either free or low cost software tools:

  1. using QuickTime Pro
    • open the .mov with QuickTime Pro and export it as .m4v  (which is what QT chooses to name it)
    • rename the .m4v file extension to .mp4   (they're essentially the same format)
  2. using Handbrake
    • open the .mov and export it as .mp4
  3. then in either case 1 or 2, use Firefogg to convert the .mp4  to .webm and .ogv in separate passes.(necessarily using Firefox)

Or, alternatively, just use Sorenson Squeeze on Mac or Windows to encode all three.

THE HTML5 CODE

This is the HTML5 <video> code used in the examples. It's a straightforward HTML5 <video> element with three <source> elements, and fallback handling for IE 7-8 provided by flowplayer. It handles IE, Firefox, Safari, Chrome, and Opera, as well as iPad and iPhone Mobile Safari and WebClips. The conditional comment launches the video in IE 7-8 which do not implement HTML5 <video> functionality.

This page uses an HTML5 <!DOCTYPE HTML> and is W3C validated. It also conditionally loads modernizer for IE 7-8 to provide HTML5 element support. Note that playing the .mp4 in flowplayer requires Flash player 9.0.115 or above. Alternatively, you could instead encode and use example.flv in the fallback code to support earlier versions of Flash.


<video id="example" width="320" height="180" controls="controls">
    <source src="video/example.webm" type="video/webm">
    <source src="video/example.ogg" type="video/ogg">
    <source src="video/example.mp4" type="video/mp4">
    <!-- the flowplayer fallback must be loaded *after* closing the /video tag -->
</video>

<!--[if lt IE 9]>
    <script type="text/javascript"> // the video fallback
       // first delete sources so that flowplayer will autoload
       document.getElementById('example').innerHTML = ''; 
       flowplayer('example', '../flash/flowplayer-3.2.7.swf', {
            clip:  {
                url: 'video/example.mp4',
                autoPlay: false,
                autoBuffering: true
                }
        });
    </script>
<![endif]-->
	 

Flowplayer has an extremely rich API and there are many ways that you can load and control it. The code above is just one example. Another common method of placing multiple players on one page is to use anchor tags (i.e. <a ...>) to install your players. See the note on autobuffering below.


Optimizations

In order to keep this code as simple as possible, I've foregone a number of optimizations I would normally make. For example:

  1. the fallabck code uses IE 'conditional comments' to invoke Flowplayer only for IE 6-8. However, older versions of other browsers including Firefox and Safari, don't support the <video> element. It would be a simple process to detect support of the <video> tag (and other error conditions), and dynamically load Flowplayer for them as well;
  2. I've placed the fallback code beneath each video instance. However since there are multiple players (3) on this page, I would normally create a single JavaScript function to configure and load the players parametrically
  3. autoBuffering: the autoBuffering parameter in the flowplayer configuration above determines whether or not the video(s) will be streamed from the server as soon as the page is loaded. Autobuffering can result in an unnecessary and significant load on the server, particularly when you have multiple videos on one page. Example 1 uses autobuffering; Examples 2 and 3 use a poster image instead. Watch the videos in IE 7 or 8 to see the difference in behavior; the tradeoff is server load vs video startup speed.
    preload: the preload="metadata" attribute of the HTML5 <video> element provides essentially the same functionality.
  4. I've left some css and script code in the <head> section of this page which I'd normally place in external css and js files. This was done so that you can see them with a "View -> Source" of this page. Otherwise, I recommend consolidating code in external css and script files to separate it from "semantic HTML".

VIDEO PLAYERS

There are any number of imperatives to provide for Flash Video (.flv file format) as the backup media particularly in view of its former popularity as a video format (that is, before Apple's 'deprecation' of .flv in its decision not to support .flv on its mobile devices). I nevertheless find myself with quite literally thousands of legacy video previews in .flv format on hand. And since it's simply patently impractical from both a cost and workflow standpoint to transcode legacy previews to three formats (H.264/mpg, .webm, and .ogg/ogv) it's typically obligatory to provide a mechanism for playing such videos.

TROUBLESHOOTING

Symptom: Video doesn't play

The most common issue I've come across is that the video doesn't play, although the <video> element is visible on the page with a large "X" in the center. This is commonly due to mime type conflicts in the http header sent from the web server. In particular, I've found that GoDaddy Economy and Premium Linux hosting (which use Apache) do not include the necessary video mime types in the Apache configuration file mime.types. This is easily corrected (for Apache — not IIS) by creating an .htaccess file as shown below, and placing it in the directory that contains the videos:

  1. Create an empty text file named ".htaccess" (note the leading dot, and no file extension)
  2. Use a plain text editor such as Notepad++, Windows Notepad, or vi (not Word!) to enter the following three lines:
    • AddType video/ogg ogg ogv
    • AddType video/webm webm
    • AddType video/mp4 mp4
  3. Save the file and upload it to the directory on your webserver that contains your videos. You're done.
Advanced Administrators: If you have access to httpd.conf and are comfortable editing Apache's configuration files, then add the directives above into http.conf instead, save it, and restart Apache. I strongly discourage modifying mime.types directly since edits made to mime.types will be overwritten the next time you upgrade Apache.