HLSL pixel shader for horizontal keystone correction

I've been working on a HLSL pixel shader to do horizontal keystone correction. Too many nights fighting with triangles and sh*tty HLSL .fx files not doing what I expect them to do!..

In memory of the first time I got to try VJing on a TH2Go setup at a VJlondon meeting at Cafe 1001, I present a sneak peek into my window with an approx 5.33:1 display running across the "long" wall (and doors...), from a single video projector.

To give you some idea of the size, I suppose it is about one and a half times as wide as the wall you are looking through the window of to see it. I mean the wall not including the door in the dark corner (so up to about where the top step is).



I'm thinking about reworking the shader to accomodate the projector being oriented vertically, thus getting back the possibility of a not-so-widescreen aspect. This will basically be the same as the existing shader with X and Y switched; a 90 degree rotation then vertical-keystone-corrected.

Projectors I've seen don't seem to allow "extreme" keystone correction, and I'm curious as to whether the correction they do is "properly" warped, or if it only adjusts the edges, as I initially did; it might be that they only allow minor "correction" because the nonlinear warping isn't so noticeable at small angles.

-spxl

good stuff

how about vertical one?

vertical + horizontal = basic mapping engine

I think arbitrary (horizontal + vertical) correction is a more complex problem.

The calculations I made depend on some assumptions:
• The projection surface is vertical
• The projector is "horizontal", so there is essentially no pitch and no roll, only yaw (rotating in the horizontal plane); by implication (note sure if this holds..): "vertical lines" in the content remain vertical lines in the output and also appear as vertical lines when viewing the screen
• The far left and far right edge in the projection match the left and right edge of the content (ie, no horizontal windowing or zooming)
[/list:u]

The horizontal correction was designed for a practical purpose - side projection at a gig last weekend - though unfortunately by that stage hadn't catered for the nonlinear horizontal distortion. My shader actually "corrects" the horizontal and vertical separately: the vertical is linear, and based purely on the appearance of the live projection - I got very nice, straight top and bottom edges, and horizontal lines appeared nice and parallel. Vertical lines also were fine, but circles and so on didn't look right.

My updated shader with horizontal distortion has calculations that depend on input values for the horizontal "field of view" (angle for the width) of the beam as well as the angle of rotation away from directly facing the wall. I wasn't able to get a circle on the left and a circle on the right of my test pattern to both look "right" at the same time in the test I did at home. However, for that test I didn't know the correct input angles, so had just been adjusting a couple of sliders (where even one of those was scaled relative to the other... so who knows what numbers I was feeding in!). I need to have another look around to see if I can find a protractor (and/or get out the tape measure some distances, since I don't so much need the angle as the tan(angle) values).

A stumbling block for vertical + horizontal mapping in one shader is that I've run out of parameters that I can pass from my software to the shader: I'm currently using 16 and that seems to be the limit. (I tried using 19 and found that the last 3 were dead...). As has happened at other times that I've played with HLSL, I've become frustrated by not understanding why some things work and some things don't, such as trying to access pixels from a second texture.\

eg

[code:1]sampler Sampler = sampler_state
{
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
};

sampler AlternateSampler = sampler_state
{
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
};[/code:1]

Using the tex2d(which_sampler, float2(posx, posy)) seems like it should just give you a colour from Sampler or AlternateSampler at (posx, posy) (where posx and posy are between 0 and 1), but in various combinations I've been frustrated repeatedly by not being able to understand why sometimes I'd get colours from my "testpattern" image (which was specified as the alternate texture) and whatever content is coming from the underlying layers in the mixer. Time to give up and look elsewhere, methinks!

-spxl

And finally, a live demo of the finished product.



http://www.youtube.com/watch?v=MX_VH8a-4kQ

-spxl

how many projectors are used here?

One projector, located off to the far right of the frame.

I also ran a test in my back yard on the new fence: about 12m wide by 1.8m high. Not the best picture quality at that size, as you can imagine, but still looked pretty cool.

-spxl

looks really good

Hello,

looks good on the screen.

for horizontal+vertical projection you can use a Homography transform.

Roughly you give 4 points (each corner of your projection area) and it gives you back the transform you need. It involves solving a "simple" linear system, there's some libraries doing that for you. If you need some links let me know, not sure which language/software you use for your visuals.

If you use directx you wouldnt even need to do it in a shader, you can just set the homography as the projection transform and it will do the job for you.

If you have more questions on that one feel free to ask

I am very interested in the links - I implemented a homography transform for a project (with Dr.Mo) and had a hard time trying to find how to do it (eg the Wikipedia article on homography has a some stuff with matrices, but really doesn't explain what to do); in the end I found some code in a panorama tools C library, ported the solving methods to C# and implemented the effect using a pixel shader; I suspected it could be possible to use the projection matrix somehow, but don't understand how it works and how to set it up.

For the Vj software I use, the transform is purely a ".fx" HLSL pixel shader file; there is no opportunity to do the linear equation solving to calculate the coefficients.

-spxl

Out of interest, what vj software are you using? And does it allows you multiple render passes (which is very handy thing to have :)


One library which solves it for you is OpenCv (cvGetPerspective). On the source code they give you the equation as well, so I implementing it in any other language is straightforward. If you want me to paste it here let me know.

Indeed, you wouldnt want to solve the equation on the pixelshader,which is possible, but performance wise solving it for every pixel is not ideal :)

Best option i guess is to have a small config program to build the homography (I normally use 4 XY sliders to move the corners), then save the output transform somewhere (either in a file, or copy paste in the shader if it's the only option). Should be easy enough in stuff like processing/of/vvvv

I'm using Neon v2. An old beta version (the newer 2.5 or whatever it is never worked properly for me), but the codebase is now open source.

I don't know if it allows multiple render passes; I haven't learned enough of HLSL to know how to use that at all yet.

I did consider the option of making a separate program that calculates the homography parameters, but haven't got around to trying that yet. :)

-spxl