The 3rd Dimension
Recently at work, my coworker Jimm Wagner thought it would be fun to see if we could accomplish creating a “live” 3d video using Silverlight 4, a couple of webcams, and some old school red and blue 3d glasses. With the new webcam support that is now available in Silverlight 4, making this work was actually very simple. The resulting 3d in the video isn’t perfect, but there is some feeling of depth once the videos are aligned properly.
Getting Started
The basics for how to accomplish this are quite simple, use the new webcam support in Silverlight 4 to get a list of available video devices, take a couple of Rectangles and fill them with a video brush, and create a couple of pixel shaders to get the colors you want. A little more detailed look at how this all goes together, plus a download of the code, will be available below.
Click here to run the demo. Note: You will need the the Silverlight 4 runtime and at least 1 webcam plugged into your computer for the demo to work. The runtime is available for both Windows and Mac.
The Pixel Shaders
To create these pretty simple pixel shaders, I utilized the Shazzam Pixel Shader Utility to create the HLSL for the desired effects. The two effects that were created were Red Only, which removed any color from the image besides the red, and a Minus Red, which removed both the red and the alpha from the input source. The code for the fx files is as follows:
RedOnly.fx
sampler2D input : register(s0);// This shader will remove the green and blue colors from the sourcefloat4 main(float2 uv : TEXCOORD) : COLOR { float4 color= tex2D( input , uv.xy); // remove green and blue color.g = 0; color.b = 0; return color;}
MinusRed.fx
sampler2D input : register(s0);// This shader will remove the red color and alpha from the sourcefloat4 main(float2 uv : TEXCOORD) : COLOR { float4 color= tex2D( input , uv.xy); // remove red and alpha color.a = 0; color.r = 0; return color;}
One handy feature in the Shazzam tool is that it will generate the postscript files as well as the .Net classes that you need to implement your pixel shaders. More info can be found on the Shazzam site.
The Silverlight
So the first thing we need to do is the get all of the available webcams that are plugged into the computer. This is handled in the main page loaded event of our main window. Once we get a list of available video cameras, we check to see how many were found. If there is only a single camera, we go head and set one of our sources to that camera. Otherwise, we will bind a couple of comboboxes to the list of our available cameras so that the sources can be set at a later time.
void MainPage_Loaded(object sender, RoutedEventArgs e){ // get available devices ReadOnlyCollection devices = CaptureDeviceConfiguration.GetAvailableVideoCaptureDevices(); _deviceCount = devices.Count; if (_deviceCount == 1) { // enable the capture and reset buttons this.btnCapture.IsEnabled = true; this.btnReset.IsEnabled = true; // disable the source selections this.btnSetSource.IsEnabled = false; this.cbRedWebCamOptions.IsEnabled = false; this.cbBlueWebCamOptions.IsEnabled = false; // check if red source is null if (_redSource == null) _redSource = new CaptureSource(); // set the red source capture device _redSource.VideoCaptureDevice = devices[0]; // check if blue source is null if (_blueSource == null) _blueSource = new CaptureSource(); } else { // bind the combo boxes this.cbRedWebCamOptions.DataContext = devices; this.cbBlueWebCamOptions.DataContext = devices; }}
So if we have more than one available video device, we will set the VideoCaptureDevice of each source in the click event of our btnSetSources button that will be enabled if more than one camera is found. We will also check to see if the same camera is chosen for both sources and will only use our _redSource if this is true.
// check if red source is nullif (_redSource == null) _redSource = new CaptureSource();// check if blue source is nullif (_blueSource == null) _blueSource = new CaptureSource();if (this.cbRedWebCamOptions.SelectedItem as VideoCaptureDevice == this.cbBlueWebCamOptions.SelectedItem as VideoCaptureDevice){ _sourcesAreSame = true; // set video capture device _redSource.VideoCaptureDevice = this.cbRedWebCamOptions.SelectedItem as VideoCaptureDevice;}else{ _sourcesAreSame = false; // set video capture device _redSource.VideoCaptureDevice = this.cbRedWebCamOptions.SelectedItem as VideoCaptureDevice; _blueSource.VideoCaptureDevice = this.cbBlueWebCamOptions.SelectedItem as VideoCaptureDevice;}
And now that we have our capture devices initialized, we can create our video brushes and start the capture. If there is only a single camera, we will set it up so that our two video brushes are using the same camera. If the camera(s) are already running, we will stop the capture and reset our rectangles.
void btnCapture_Click(object sender, RoutedEventArgs e){ try { if (_redSource.State != CaptureState.Started && _blueSource.State != CaptureState.Started) { // make sure sources are stopped _redSource.Stop(); _blueSource.Stop(); // create red video brush if null if (_redBrush == null) { _redBrush = new VideoBrush(); _redBrush.Stretch = Stretch.Uniform; } // create blue video brush if null if (_blueBrush == null) { _blueBrush = new VideoBrush(); _blueBrush.Stretch = Stretch.Uniform; } // check the number of devices if (_deviceCount == 1 || _sourcesAreSame) { // set the red video brush source and rectangle fill _redBrush.SetSource(_redSource); this.RedRectangle.Fill = _redBrush; // set the blue video brush source and rectangle fill // since there is only one camera, both of the video brushes source will be the same blueBrush.SetSource(_redSource); this.BlueRectangle.Fill = _blueBrush; // we offset the red rectangle to try and create the 3d effect for only one camera this.sldRedHorizontal.Value = -5; // ask user for permission if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess()) { _redSource.Start(); } } else { // set the red video brush source and rectangle fill _redBrush.SetSource(_redSource); this.RedRectangle.Fill = _redBrush; // set the blue video brush source and rectangle fill _blueBrush.SetSource(_blueSource); this.BlueRectangle.Fill = _blueBrush; // ask user for permission if (CaptureDeviceConfiguration.AllowedDeviceAccess || CaptureDeviceConfiguration.RequestDeviceAccess()) { _redSource.Start(); _blueSource.Start(); } } // change capture button content this.btnCapture.Content = "Stop"; } else { // stop input sources _redSource.Stop(); _blueSource.Stop(); // remove the rectangles fill this.RedRectangle.Fill = null; this.BlueRectangle.Fill = null; // reset capture button content this.btnCapture.Content = "Capture"; // reset the rectangles ResetRectangles(); } } catch (Exception ex) { MessageBox.Show(ex.Message, "Error using webcam", MessageBoxButton.OK); }}
The final piece to the puzzle is using the pixel shaders that we created and compiled above. We add them in the xaml by adding in the following namespace xmlns:shaders=“clr-namespace:WebCam3d.Shaders” and then referencing it as follows:
<!-- On our Red Rectangle --><Rectangle.Effect> <shaders:RedOnlyEffect /></Rectangle.Effect><!-- On our Blue Rectangle --><Rectangle.Effect> <shaders:MinusRedEffect /></Rectangle.Effect>
Finally, I added in a couple of horizontal and vertical sliders that allow you to shift the two rectangles to get them lined up correctly for the 3d effect. This helps when using two different model webcams or just a single camera for input, plus the fact that my rig took slightly less time to develop than this one.
Conclusion
Hopefully this post was able to show how easy it is to implement multiple webcams in Silverlight 4 and add some simple effects to them as well. I am relatively new to Silverlight, but was still able to get a working demo up and running in a short amount of time using these techniques. Any comments or questions and suggestions on how this could be made better are more than welcome. Happy coding!
Code Download
Visual Studio 2010 RC Solution: WebCam3d.zip (78.00 kb)