Greenscreen code and hints

The following is a greenscreen algorithm that has worked for me. It is technically a chromakey program, working with any color background.

pseudo code for the algorithm

chromakey algorithm input: fg(i,j) = i,j element of forground bg(i,j) = i,j element of background tola, tolb returns: out(i,j) get Cb and Cr of key color; call them Cb_key and Cr_key for each i,j: get Cb and Cr for pixel value; call them Cb_p and Cr_p let mask = colorclose(Cb_p, Cr_p, Cb_key, Cr_key, tola, tolb) let mask = 1.0 - mask out(i,j) = fg(i,j) - mask*key_color + bg(i,j)*mask def colorclose(Cb_p, Cr_p, Cb_key, Cr_key, tola, tolb) temp = math.sqrt((Cb_key-Cb_p)**2+(Cr_key-Cr_p)**2) if temp < tola: return 0.0 else if temp < tolb: return (temp-tola)/(tolb-tola) else: return 1.0

Description of the algorithm

The basic idea is that we convert our images into YCbCr and look at the CbCr plane (we ignore luminesence). The key color defines a point on the plane. We divide the plane into three regions a region close to the key color, a region far from the key color, and a middle region. The close region is all colors that are less than tola from the key color. the far region is all colors farther than tolb from the key color. The middle region is the colors inbetween tola and tolb. We generate a mask as follow, the near region is all background (0), the far region is all foreground (1). The middle region is assigned a value based on its distance from the key color. we use the function (temp-tola)/(tolb-tola) where temp is the distance to the key color. (this function is 1 at distance = tola and 0 at distance = tolb.)

The neastest part of this algorthim, and one I borrowed from from Ashihkmin 1997 (available at http://bmrc.berkeley.edu/courseware/cs294-3/fall97/projects/ashikhmin/) is to remove the key color from the foreground so that you get less green halo around the forground objects. I have had remarkable results on really poor imput data. I got usuable results by standing in front of my pale green wall that was lit from the bottom. I have had really good results using a special very green cloth in the evening (even light on the screen).

C code

The below code is a C implementation of this algorithm. It is very simple, and uses windows bmp files for input and output. The strange format of the bmp file somewhat obscures the logic, but all of the stuff is there. In AOI aware Java the looping would be much better, and the images are already im memory, this version was designed to use almost no memory (only 1 pixel at a time is in memory) and have no external dependancies. It also was my first C program in like 7 years.

/*a fast implementation of the chromakey program (c) 2008 Edward Cannon feel free to use or modify at will*/ #include #include int max(int a, int b) { if (a>b) {return (a);} return (b); } /*the follwing three functions convert RGB into YCbCr in the same manner as in JPEG images*/ int rgb2y (int r, int g, int b) { /*a utility function to convert colors in RGB into YCbCr*/ int y; y = (int) round(0.299*r + 0.587*g + 0.114*b); return (y); } int rgb2cb (int r, int g, int b) { /*a utility function to convert colors in RGB into YCbCr*/ int cb; cb = (int) round(128 + -0.168736*r - 0.331264*g + 0.5*b); return (cb); } int rgb2cr (int r, int g, int b) { /*a utility function to convert colors in RGB into YCbCr*/ int cr; cr = (int) round(128 + 0.5*r - 0.418688*g - 0.081312*b); return (cr); } double colorclose(int Cb_p,int Cr_p,int Cb_key,int Cr_key,int tola,int tolb) { /*decides if a color is close to the specified hue*/ double temp = sqrt((Cb_key-Cb_p)*(Cb_key-Cb_p)+(Cr_key-Cr_p)*(Cr_key-Cr_p)); if (temp < tola) {return (0.0);} if (temp < tolb) {return ((temp-tola)/(tolb-tola));} return (1.0); } int get32(FILE * f) { /*reads a 32 bit int from file*/ int a, b, c, d; a = getc(f); b = getc(f); c = getc(f); d = getc(f); /*printf("get32 %i %i %i %i ", a, b, c, d);*/ return (a + 256*b + 65536*c + 16777216*d); } struct bmp { int valid; int filesize; int datapos; int width; int height; FILE * file; }; struct bmp readBMPheader(const char * filename) { /*reads the BMP header and decides if everything is ok, returns a struct bmp*/ FILE * file; char b, m; int in; struct bmp thisfile; file = fopen(filename, "rb"); if (file == NULL) { thisfile.valid = 0; return (thisfile); } /*check magic number*/ b = (char) getc(file); m = (char) getc(file); /*printf("test");*/ if (b != 'B' || m != 'M') { thisfile.valid=0; return (thisfile); } /*printf("%c%c", b, m);*/ thisfile.valid = 1; thisfile.file = file; thisfile.filesize = get32(file); /*skip past reserved section*/ getc(file); getc(file); getc(file); getc(file); thisfile.datapos = get32(file); /*get width and height from fixed positions*/ fseek(file, 18, SEEK_SET); thisfile.width = get32(file); thisfile.height = get32(file); return (thisfile); } struct bmp writeBMP(const char * filename, struct bmp example) { /*returns a struct bmp with the header written and its file open for writting data copies header information from example, makes file right size filled with black*/ int i; struct bmp thisfile; thisfile.file = fopen(filename, "wb"); if (thisfile.file == NULL) { thisfile.valid = 0; return (thisfile); } fseek(example.file, 0, SEEK_SET); for(i=0;i { fputc(getc(example.file), thisfile.file); } for(i=0;i { fputc(0, thisfile.file); } return (thisfile); } int main (int argc, char *argv[]) { /*must be 8 command line arguments filenames fg, bg, out, r, g, b, tola, tolb*/ if (argc != 9) { printf("must be 8 command line arguments\n"); printf(" fg bg out r g b tola tolb"); return(1); } int b, g, r, y, cb, cr, r_bg, g_bg, b_bg; int b_key, g_key, r_key, cb_key, cr_key, tola, tolb; int pos, mx; double mask; struct bmp fg, bg, out; /*set up for chromakey */ fg = readBMPheader(argv[1]); bg = readBMPheader(argv[2]); out = writeBMP(argv[3], fg); r_key = atoi(argv[4]); g_key = atoi(argv[5]); b_key = atoi(argv[6]); cb_key = rgb2cb(r_key, g_key, b_key); cr_key = rgb2cr(r_key, g_key, b_key); tola = atoi(argv[7]); tolb = atoi(argv[8]); /*loop through file and preform chromakey*/ pos = fg.datapos; mx = 3*fg.width; switch (mx%4) { case 1: mx = mx+3; break; case 2: mx = mx+2; break; case 3: mx = mx+1; break; } int i, j; for (i=0; i { fseek(fg.file, pos, SEEK_SET); fseek(out.file, pos, SEEK_SET); fseek(bg.file, pos, SEEK_SET); for (j=pos; j { b = getc(fg.file); g = getc(fg.file); r = getc(fg.file); b_bg = getc(bg.file); g_bg = getc(bg.file); r_bg = getc(bg.file); cb = rgb2cb(r,g,b); cr = rgb2cr(r,g,b); mask = colorclose(cb, cr, cb_key, cr_key, tola, tolb); mask = 1 - mask; r = max(r - mask*r_key, 0) + mask*r_bg; g = max(g - mask*g_key, 0) + mask*g_bg; b = max(b - mask*b_key, 0) + mask*b_bg; fputc(b, out.file); fputc(g, out.file); fputc(r, out.file); } pos = mx+pos; /*printf("\n");*/ } /*close files and clean up*/ fclose(fg.file); fclose(bg.file); fclose(out.file); return (0); } A python impemintation is also available, but it is highly optimized for speed and is almost unreadable.