I've recently been playing around with recording TV shows under Linux and converting the saved video to XViD format so I can watch it later using the TV-Out function on my laptop. It took a while to get to grips with all the new technology involved, so I thought it would be useful to document my efforts here.

Bearing in mind that apart from my amateur radio licence, (which expired many more years ago than I care to admit!) I have no qualifications in the field of electronics, click here for a potted explanation of how analog and digital TV works. Hopefully it's not too far from the truth, but best take it with a small pinch of salt just to be on the safe side... :-)

There are several excellent resource on the Internet for anyone who wants a detailed technical explanation of how Digital Video Broadcasting, or DVB, works. In a nutshell, there are 3 main standards. Regular satellite TV (DVB-S), cable TV (DVB-C) and digital terrestrial TV, known as DVB-T. It is only DVB-T that I'm conerned with here. Some of the stuff on re-encoding to XViD may still apply, but I would recommend you carry on Googling!

There are a whole bunch of different DVB-T add-on cards and devices which have very good Linux support. Finding one which suits your needs will be an entirely personal thing, so I won't attempt to cover that in this article. Suffice it to say that you'll need to do the usual research to establish that any hardware you plan to purchase will work with your chosen distro.

Based on the success that a work colleague reported, I invested a small amount of money in a Freecom DVB-T USB stick. This device is a 'budget' DVB-T receiver. This means that it relies on external software to do the job of decoding the MPEG data stream and selecting the required PIDs. It comes with a reasonably effective miniature antenna and a remote control which I haven't bothered to try and do anything with in Linux. The picture over there shows what it all looks like.

I quickly discovered that the kernel on my laptop was too out of date to be able to support the Freecom device. To be fair, the kernel was able to support the rev1 and rev2 versions, but the one I bought turned out to be a newer rev3 device. This kind of thing is common with proprietary hardware - the manufacturer will start producing a new version under the same brand name, but using different hardware internally. The USB id for this thing is 14aa-0225 for the benefit of Googles web spider! I put myself through the pain of updating the Linux install on my laptop to begin with, as I needed a kernel revision of 2.6.13 or higher in order to be able to make it work. I installed Mandriva 2007, which replaced my previous Mandrake 2005LE installation which I'd had in place for some time. There was a fair bit of tedious hacking around to get everything working properly, click here for the full story.

By this point, your DVB stick works using kaffeine, the media player which is commonly bundled with KDE. If that's all you ever want to do, you can stop here. For more advanced stuff, read on...

The next step is to generate a channel list in a format which is compatible with other DVB-aware applications such as mplayer and xine. You should already know the name of your local TV transmitter, having used this information to get kaffeine working. What you now need is a list of the UHF frequencies that your local transmitter uses. Happily, KDE already provides this information for you. Take a peek into:

/home/yourusername/.kde/share/apps/kaffeine/dvb-t

You will see a bunch of files in there, some of which start with the prefix uk. In my case, I live near the Whitehawk Hill transmitter in Brighton. The file I used was therefore:

/home/yourusername/.kde/share/apps/kaffeine/dvb-t/uk-WhitehawkHill

For the truly bored, the contents of the file look something like this:

# Whitehawk Hill, East Sussex (Brighton)
# T freq bw fec_hi fec_lo mod transmission-mode guard-interval hierarchy
# multiplex B BBC - Channel 48
T 690000000 8MHz 3/4 NONE QAM16 2k 1/32 NONE
#  multiplex 2 Digital 3&4 - Channel 50
T 706000000 8MHz 3/4 NONE QAM64 2k 1/32 NONE
# multiplex A SDN - Channel 55
T 746000000 8MHz 3/4 NONE QAM64 2k 1/32 NONE
# multiplex C Crown Castle - Channel 58
T 770000000 8MHz 3/4 NONE QAM16 2k 1/32 NONE
# multiplex D Crown Castle - Channel 61
T 794000000 8MHz 3/4 NONE QAM16 2k 1/32 NONE
# multiplex 1 BBC - Channel 66
T 834000000 8MHz 3/4 NONE QAM16 2k 1/32 NONE

You need to get your DVB device to tune each of these frequencies in turn to see what digital data streams it can find. This is done using a command line tool, scandvb, which is provided as part of the dvb-apps RPM. The actual commands you need to run are:

cd /home/yourusername
mkdir .tzap
cd /home/yourusername/.kde/share/apps/kaffeine/dvb-t
scandvb uk-WhitehawkHill > /home/yourusername/.tzap/channels.conf

This will take a few minutes and you will end up with a file containing all the digital data streams that could be found. Note that I deliberately used the phrase digital data streams - this is because the file you just generated also contains details of any audio-only services that were found. You can safely edit these out of the channels.conf file using your favourite text editor.

There is one other slight wrinkle we need to take care of...this being the UK, nothing is ever simple. The range of digital channels being broadcast actually changes at various times during the day, most notably at 6:00pm when the childrens channels go off the air and again at 7:00pm when (for example) BBC3 starts broadcasting. There may be a way to get the Freecom to detect this, but the only solution I was able to come up with was to run several seperate channel scans at different times of day and combine the resulting channels.conf files into a single master list. Tedious in the extreme, but there you go...

In order to finally make everything work with xine and mplayer, you need to either copy or symlink your master channels.conf file into the relevant config directories. For example, I keep my master channels list in:

/home/phile/dvb/channels.conf

Therefore to make this available to the other apps, I did this:

cd /home/phile
mkdir .xine
mkdir .mplayer
cd .xine
ln -s /home/phile/dvb/channels.conf
cd ../.mplayer
ln -s /home/phile/dvb/channels.conf

You should, of course, replace /home/phile with the path to your own home directory.

As explained earlier, DVB data streams are essentially the same as the video/audio data you would expect to find on a standard DVD. Video DVD is a mature and established technology platform, so I was slightly surprised to discover that most of the normal tools you would use on Linux for dealing with DVD-style data seem to have problems when trying to cope with digital TV. It turns out there are a number of reasons for this:

• UK DVB transmissions aren't always strictly compliant with MPEG-2 standards
• Source material can have a number of different framerates and this has to be modified to comply with the 25fps PAL TV standard, meaning duplicate frames are sometimes added to keep audio and video in sync.
• Frames are sometimes dropped for the same reason as above.
• Good old fashioned interference can cause audio dropout or video breakup, expecially on misty days with a weak TV signal.

My initial experiments were based around using mencoder to do the recording. Whilst this is a superb piece of software, it's simply too clever by far for this application. What is needed is something which can handle the job of tuning to the required UHF channel, selecting the correct video and audio information from amongst the many multiplexed data streams being broadcast and simply dump this information to disk as a regular MPEG a/v file. Any other processing which is required can be carried out at a later stage. The program required for this is dvbstream, which is normally supplied as part of the DVB-tools package on Mandriva.

As a working example, in order to record 15 minutes of whatever programme is currently showing on BBC3, the command would be:

/usr/bin/dvbstream -f 834000 -ps 620 621 -n 900 \
    -o > /home/phile/dvb/recordings/BBC3.mpeg

The various options given are as follows and all are explained in more detail if you type dvbstream on its own at a shell prompt:

-f 834000 specifies the base UHF frequency in kHz that we want our DVB card to be tuned to. Note that the normal channels.conf file will show this value in Hz, so you have to remember to drop the last 3 zeroes off the end.

-ps 620 621 specifies the DBV programme ids for the video and audio stream that we're interested in. This information will be found in your channels.conf file.

-n 900 sets the recording duration in seconds.

-o > /home/phile/dvb/recordings/BBC3.mpeg specifies that dvbstream should stream the data to stdout, which we then redirect to a local file.

Most digital TV broadcasts have bitrates equivalent to DVD quality. You can reduce the bitrates quite considerably without impacting the overall quality too much, saving yourself a ton of disk space in the process. I looked at several different ways of doing this before settling on a fairly simple calculation which so far has given me excellent results.

Let's take as an example a recording that I made of the Doctor Who Christmas special in December 2006. My recording started at 19:55 and is 4200 seconds long, i.e. 1 hour and 10 minutes. This includes a certain amount of slack time at the start and end of the actual programme, but I'm not going to worry about trimming this out just yet. The estimated final output size for this will be:

(4200 / 60) * 6000 = 420000

This is simply the number of minutes of recorded material multiplied by 6000 to give a value in kbytes. The reason for picking this value will become clear if you look at the answer given for a hypothetical 90 minute recording that needs no trimming:

90 * 6000 = 540000

This would give a final output size which allows a normal 90 minute movie to easily fit onto a single blank CD for archival purposes. The required mencoder command line to convert my Doctor Who recording to XViD is:

/usr/bin/mencoder /home/phile/dvb/recordings/drwho.mpeg -of avi \
   	-ovc xvid -xvidencopts bitrate=-420000:par=pal169 \
	-oac mp3lame -lameopts cbr:br=128 -audio-delay -0.3 \
	-vf pp=fd \
	-af volnorm=1:1.10 \
	-mc 0 -noskip -o /home/phile/dvb/xvid/drwho.avi

As with recording, please refer to the man page for mencoder to see a full explanation of all these options, but in summary they are:

-of avi specifies output in AVI format.

-ovc xvid specifies XViD as the output video codec.

-xvidencopts bitrate=-420000:par=pal169 specifies a final output size of 420Mbytes and a 16:9 (i.e. widescreen) pixel aspect ratio. Substitute the correct size for the file you are encoding using the calculation above.

-oac mp3lame specifies mp3lame as the audio output codec.

-lameopts cbr:br=128 sets the audio to be constant bit rate at 128kbps.

-audio-delay -0.3 delays the audo track by 0.3 seconds, which is just the right amount to keep audio and video in sync for a UK DVB-T broadcast.

-vf pp=fd adds a video de-interlace filter to make the output slightly smoother.

-af volnorm=1:1.10 normalizes the audio level by giving the quiet bits a very slight boost.

-mc 0 Disable mencoders automatic attempts to re-sync audio with video.

-noskip Do not drop duplicate frames.

-o /home/phile/dvb/xvid/drwho.avi specifies where to save the final XViD file.

I quickly got bored with trying to remember all these command line options and other mental acrobatics, so I wrote a Python script to take care of it all for me. To download a copy of this, right click this link and select save link as... in your browser. Ensure you have a directory called bin in your home directory and save the script there. Make it executable using the command chmod 755 /home/yourusername/bin/dvbUtils.py. Then run dvbutils.py install to set up the required symlinks for the recording and encoding functions. Finally, you need to run dvbconfig to complete the setup.

You can now run dvbrecord to record a programme, dvbencode to convert a saved recording from MPEG to XViD format and dvbconfig if you wish to change the setup. I'm working on an extension to this script which will add a dvbschedule command so that recordings can be scheduled in advance - please check back to this page shortly for an update.

Now that you have a saved XViD file, this is a good time to trim the beginning and end of the recording to match the exact length of the programme. Some TV stations will insist on inserting an all-too-frequent 'message from our sponsors' into the middle of your favourite show, so you'll most likely want to get rid of those as well. My experience has taught me that it's best to do the encoding to XViD before trying to do any trimming, to avoid stuffing up the audio sync in your final output file, hence why I've left tis part until last.

By far the best tool I've found for this job is avidemux - this comes in RPM format for Mandriva 2007, you may need to do some digging to find a version for your chosen distro.