How to get stills into video

13 replies [Last post]
Alan Roberts at work
Offline
Joined: May 6 1999

I've opened the can of worms, and I fully expect you lot to come back at me with a vengence, because, as usual, this is a big topic. I can't cover all of it in one message (time constraints), and some of it would be better with ilustrations, but I can't help that. Anyway here goes.

MY SETUP IS:

Pansonic DX100 camcorder (DV)
Olympus C3030 zoom digital camera

PC, Dell PII/450 with:
extra hard drives for video (2xIBM25Meg on a RAID card)
Epson Stylus Phot 750 6 colour printer
UMAX scanner
Sofware:
Paint Shop Pro
Photo Shop
Premiere 5.1c
EditDV 2
and assorted other bits and pieces.

So, my problem is how to get digital stills from the camera into video, get scans of prints into video, and print digital stills.

VIDEO RESOLUTION AND FORMAT

Video resolution (DV) is 720 pixels by 576 and isn't 4:3. A 4:3 image occupies only 702 of those pixels, the outer 9 from each side are technically called "overscan" and aren't part of the video stream. Also, technically, only 575 lines of video are used, a half line is thrown away from the first and last line of the frame, so that one field starts, and the other ends, in the middle of a line. The reasons for all this are bound up in the history of the development of television, are there's not much future in crying over that. The standards are published by the ITU (International Telecommunications Union), particularly ITU-R BT.470 covers analogue transmission world-wide (the R means Recommendation, BT stands for Broadcast Television) and ITU-R BT.601 covers digitised video, for internal exchange between equipment in studios or between studio centres (it is now enshrined in MPEG2 as well, but more of that anon).

So, when getting stills into video, they have to conform. That means in both resolution and signal format. The resolution of video is rather low by photo standards, and we can get away with it because the opictures move most of the time, and we tend to track moving objects and pay little attention to the rest of the picture. Stills, on the other hand, will be pored over (not poured over as I read in the Guardian last week) and so all parts of the image must bear inspection. So, the rules for taking pictures are not the same as for taking video. But that's another story and belongs in the "Techniques" forum.

Cut to the chase.

HOW I DO IT

So I've got a still picture from my digital camera and I want to use it in video. The process has two major components:
1 reduce the resolution to that of video
2 make the signal confrom to video
Most video editors will do the former, few will do the latter.

REDUCE THE RESOLUTION

Resolution reduction is a filtering operation that any decent graphics package can make a reasonable stab at. Simply import the image into it and resize it to 720*576. But life's not that easy since the pixels of tv aren't square (702/576=1.21875, and not 1.3333...), so I've got to change the pixel shape as well. The best way is to work to a basic shape of 788*576 (no, that's not a typo, I meant 788), inside which is the required 4:3 image of 768*576, and finally squeeze it down to 720*576 for importing into video.

Both Photo Shop and Paint Shop Pro do this quite well, a suitable mixture of cropping the original image and then resizing will get the right aspect ratio. Photo Shop does a marginally better job, but you'll be hard pressed to see the difference, and then only with test pictures.

SIGNAL LEVEL CONFORMANCE

To make the image conform to video rules (and I'm talking of DV in it's native form here) I have to change the dynamic range as well. Graphics images, when stored in 8-bit form, always set black at level zero and white at 255, but tv's different. To allow for inaccurate setup of kit, video black is set at level 16 and white is at 235, only 219 levels used. The reasons for this are extremely good and shouldn't be ignored.

Paint Shop is very good for this, it has a Custom Filter, with a grid of 9 numbers, an offest and a scaler. I set the central number in the grid to 219 (the number of levels I want to end up with), the scaler to 255 (so that it multiply every pixel value by 219/255), and the offset to 16 (to add 16 to every pixel). And the levels are now correct for video. Photo Shop can probaly do it just as well if not better.

PROBLEMS

So, now I've got my still the right shape, and to the right levels. But the result might still not look nice. It certainly won't on a PC, because the contrast range will look wrong, all washed out and lacking in sparkle. However, if I've got a lot of detail in the original image, it can cause what's known as "interlace twitter" when imported into video. This is because tv does not expect every line and every pixel to be capable of being set to any level without knowledge of neighbouring lines/pixels.

BANDWIDTH, RESOLUTION, SAMPLING

The camera has a prefilter between the object (the scene) and the image (on the ccd). On low-cost cameras, it's just the lens, on high end cameras there's an extra optical element put in specifically to perform spatial filtering (birefringent, look it up in an encyclopedia). This limits the amplitude of, or completely removes from the image, those high frequencies that cause problems.

Let's look at horizontals first. With 720 pixel sites, it's tempting to say that we can resolve 720/2=360 cycles of resolution. And that would be true is the samples (pixels) were accuratelt and consistently timed to occur at the peaks and troughs of the sine-wave. But they can equally occur at the zero crossings, where the digital output would be all grey levels. So, we must allow there to be more than 2 samples (pixels) per cycle of resolution. 3 is a good number, more is better. So the resolution limit is more like 720/3=240 cycles per picture width. Trying to get more resolution than this will tend to result in "alias" components. This is what happens when there are less than 2 samples (pixels) per cycle. Under these conditions the spectral contents of the signal are reversed, centred on the sampling frequency, hard to explain in words, easier in pictures. This process, that of misrepresenting one frequency for another, can cause havoc in an image, less so in static ones. It is the cause of wheels going backwards (remember the stage coach wheels in John Wayne pictures?) when temporal sampling is too slow for the motion.

Vertically, the problem's more subtle. Although we have 576 frame lines to put information on, only half of them are there in each field. The frame (25 per second) is a record of two separate images, one 1/50th second after the other. So the only really belong together when the image is stationary and there's little difference between adjacent frames. When there's motion, there's a problem.

When the image is moving, we have to think of each field as standing alone. Thus there's only 288 lines to think of. So the vertical resolution is 576/2=288 cycles per picture height when static, 288/2=144 when moving. In practice, we tend to get away with something in between, and this gives rise to "interlace twitter".

TWITTER

Any vertical frequency present in the scene between 144 and 288 cycles/height will cause twitter. Since the fields come at us at 50Hz, but the frames are at 25Hz, any frequency bewteen 144 and 288 belongs to the frame and so is presented to us at 25Hz, and is visible as a slow twitter. Lower frequencies, below 144 belong to the fields and come to us at 50Hz, seeming steady.

So what?

I have to make sure that my still won't twitter, and that means a filter. Paint Shop and Phot Shop have similar filters (so does Premiere and EditDV etc). They are the one with a 3X3 grid of numbers again. Only this time we don't want to change the signal levels. In Paint Shop, in the extreme, I use:

0 1 0
0 1 0
0 1 0

which adds together, equally weighted, contributions from the centre pixel (the one we're writing to the new, filtered image) and those above and below. I put 3 in the scaler, so that the signal level isn't change, and all the nasty vertical high frequencies go away, and my still won't twitter. Photo Shop is easier, put 0.33333... in the centre column so that they add to 1.

SCANNED IMAGES

All the above applies, but now I've got the vagaries of the print and scanner to contend with. Usually, the black level is high and there's a colour cast after scanning. Again, Paint Shop and Photo Shop can deal with it, so can Photo DeLuxe etc. I normally use Paint Shop, and keep an eye on the RGB histogramme view. It shows the distribution of levels in the image. Then I can tweak Contrast and Brightness to get it to look right on the PC. But, those controls in graphics packages don't work the same way as in video. Video Brightness slides the whole signal up/down by adding to every level, and Contrast changes the range above black level. Graphics Contrast changes the range from centre level (128) and Brightness moves that. So there's a lot of swearing and readjusting before I'm happy. An easier way is to use the 3x3 filter yet again to add/subtract a number to move the black level to about 16, and tweak the multiplier to get the peak white to about 235. Then straight into video.

Enough for now, I've missed out huge chunks, and could have put a lot of this much better, but it's a start.

Now, anybody got any questions?

------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.

Carol Vorderman
Offline
Joined: Oct 13 2000

Err?...Could you repeat that?

Carol

Alan Roberts
Alan Roberts's picture
Offline
Joined: May 3 1999

Certainly.

That.

Get my test cards document, and cards for 625, 525, 720 and 1080. Thanks to Gavin Gration for hosting them.
Camera settings documents are held by Daniel Browning and at the EBU
My book, 'Circles of Confusion' is available here.
Also EBU Tech.3335 tells how to test cameras, and R.118 tells how to use the results.

Carol Vorderman
Offline
Joined: Oct 13 2000

Alan,

I noticed that you have two registered names: Alan Roberts at work and just plain Alan Roberts...Any reason?

Alan Roberts at work
Offline
Joined: May 6 1999

Yes, one is for when I'm at work, the other isn't. I would have thought that, with your much advertised superior intelligence, you could have worked that out. Or, are you an impostor (shock, horror)?

------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.

Carol Vorderman
Offline
Joined: Oct 13 2000

Alan,

I'm at work when I'm at home and at home when I'm at work...What's the difference!

RichardJ
Offline
Joined: May 7 1999

(Skipping over the banter here ... )

I can't argue with your descriptions of colour adjustments, you're obviously far more qualified than me on the topic. However, I'm not sure about the generalities of re-sizing.

I do understand all about rectangular pixels, it's something I've been researching myself recently, and some editors will do the re-scaling for you. My expertise is with MediaStudio, and there, if you drop an image on the timeline with no positioning controls it is scaled to fit the video frame. This means that if you use an image simply cropped to 4:3, it will be scaled to fit, and so end up the correct proportion (or to allow for DV's overscan, cropped to approx 4.09:3).

I assume from what you say that Premiere doesn't do this, but just maps an image pixel-for-pixel onto the video frame, cropping and/or letterboxing as required?

I also know that EditDV handles physical aspect ratio (it "knows" the viewable format is 4:3), so should do appropriate re-scaling itself. I don't know if it is aware of the DV overscan though, it would be a shame if not, because it would still be wrong! (If anyone can tell me more about EditDV's behaviour here I'd be interested to know)

I've heard some comments that Premiere supports physical aspect ratio, but I don't use it so I'm not sure - can you enlighten me?

On the basis that the fewer times an images is scaled or transformed the better, isn't it preferable to let the video editor handle aspect-ratio compensation where possible, rather than try to tweak the pixel sizes by pre-processing in an image editor?

Finally, how accurate do you really need to be with aspect ratio? Personally I think it's easier to treat 30 fps & 25 fps DV as the same, with 8 pixels overscan each side. It's exact for 30 fps, but although 25 fps is approx. 9 each side it's not strictly speaking even a whole number! I also doubt most TVs are exactly 4:3, do you know what the typical error is?

And how many people notice or care - judging by the typical setup of widescreen TVs in Dixons and the like, viewers couldn't give a ****

Fun subject - if you like that sort of thing

------------------
Richard Jones, http://www.activeservice.co.uk
Home of the MediaStudio Pro Tutorial

Richard Jones, http://www.activeservice.co.uk
Home of the MediaStudio Pro Tutorial - Edition 3 for MSP 7

Alan Roberts at work
Offline
Joined: May 6 1999

I use both Premiere and EditDV.

Premiere does not know (version 5.1c) about non-square pixels (fixed in v6, they6 say) whereas EditDV does. Neither knows that the 576-line 4:3 image is 702 pixels wide, both take the full image width of 720 pixels. My DX100 makes 708 pixels wide, a DS11 I borrowed for a few days did 712, a DS35 did the full 720.

The NTSC image is actually 708 pixels wide at the centre of line-blanking tolerance. So, NTSC (30 or 29.97...fps) should lose 6 pixels either side and 625/50 should loes 9 pixels either side.

All this is really academic until you try to rotate one object against another and findnthat it changes size as it rotates. But it can be important when you try to sell your footage to a broadcaster who might insist on it being "broadcast-safe". However, the correct approach then is for you to say that your images accord with Rec.601 and it's up to him to blank them for analogue transmission.

P.S. I don't like resorting to banter, but lately there are some new contributors to this board who seem to want to turn everything into a joke. At least one has been thrown off by Bob, there are others who, perhaps, also qualify.

------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.

Alan Roberts at work
Offline
Joined: May 6 1999

Overscan in displays is a nightmare. And it changes with time as newer tvs are announced. There are guidleines, but they only reflect what the tvs actually do, there's no specification for overscan.

It was originally used to cover up for the manufacturers' inability to make stable scan amplifies and eht supplies. Since all tv sets did it, broadcasters used the fact to allow for sloppy production techniques (allowing mic booms in shot, that sort of thing). Fortunately, more care is now taken so the raw footage is better controlled and we don't often get that sort of problem. But we still have to be aware of overscan, just because the tvs all do it. So "action-safe" and caption-safe" zones have to be defined, from measurements on real tvs. Recent measurements on some wide-screen sets showed that the overscan is different on the internally received signals, on PAL via SCART on on RGB via SCART. I suppose it's just that they don't take account of the different delays through the circuitry.

Current view is that about 8% is lost (about 4% from each edge all round). With tvs getting more stable, and flat panles that don't change scan size with power fluctuation or image brightness, it would be nice to think that overscan would go away, but the mkers seem determined to keep it. I've seen many flat plasma panels that had more than the normal-ish 4% all round, up to 9% in extreme cases.

Like I said, overscan is a nightmare.

------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.

tom hardwick
Offline
Joined: Apr 8 1999

I haven't checked it in an A/B test Alan, but can we assume that if you watch a 4:3 transmission on a 16:9 set than there'll be no overscan on the two vertical edges of the picture? After all, they'll be positioned within the screen and not hidden by the plastic masking.

tom.

Alan Roberts at work
Offline
Joined: May 6 1999

It depends on the set. I've seen some that show the full 702 (remember that 16x9 should be 960 wide if the pixels are the same shape as for 4x3), some that can be tweaked to show the full 720 using engineering menus, and some that blank to the "standard" overscan setting. That last one is the nightmare scenario because the pictures then look square since the display overscan crops some off the top and bottom.

I just wish they'd let us see all the picture, and give us controls to set our own level of overscan, rather than assume that we all need it and provide it willy-nilly.

------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.

RichardJ
Offline
Joined: May 7 1999

Alan - I got a lot of my numbers from this page: http://reality.sgi.com/cpirazzi_engr/lg/pixelaspect.html . Maybe you know these pages anyway, he does appear to know what he's talking about.

Some of his figures disagree a bit with yours - any comments?

------------------
Richard Jones, http://www.activeservice.co.uk
Home of the MediaStudio Pro Tutorial

Richard Jones, http://www.activeservice.co.uk
Home of the MediaStudio Pro Tutorial - Edition 3 for MSP 7

Alan Roberts at work
Offline
Joined: May 6 1999

Richard,

You know me by now, I don't get into arguments. I measure and report. The EBU has set guidleines for action-safe and caption-safe boundaries derived from measurements of retail tv sets, and those measurememnts are mde by the broadcasters (mainly). So the "standards" are always a little behind the times. The last set of measurements I know of were made about 14 months ago and resulted in a modified set of internal instructions for a major UK broadcaster. Those data will get into the EBU soon, I expect. Don't ask me how I know this 'cos I won't tell.

Since we're talking of overscan in terms of what the displays do, there is inevitably a lot of room for interpretation of statistics, so numbers will vary a little depending on who does the analysis. We should not take any of the figures as hard and fast specifications. They are all based on statisics, and are results, not requirements.

------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.

Alan Roberts at work
Offline
Joined: May 6 1999

Richard, I just looked at that site. It's basically US, so the numbers should be different, they are based on tv sets made by different manufcaturers from Europe and the UK.

BUT, it contains a reference to material from Charles Poynton, and that's a good opener. He's a long-standing friend of mine. His web sites are extremely good, containing very accurate maths and information (checked by me and others :) and well worth spending time looking at.

------------------
alan@mugswellvillage.freeserve.co.uk. Delete village for a spam-free diet.