Kinect Sdk

| 15-09-2021
[RAND-1-100] Comments

Kinect v2 Examples with MS-SDK is a set of Kinect-v2 (aka ‘Kinect for Xbox One’) examples that use several major scripts, grouped in one folder. The package contains over thirty demo scenes. Please look at my new Azure Kinect Examples for Unity asset, as well. It works with the Azure Kinect (aka Kinect-for-Azure, K4A), as well. The Kinect for Windows Software Development Kit (SDK) 2.0 enables developers to create applications that support gesture and voice recognition, using Kinect sensor technology on computers running Windows 8, Windows 8.1, and Windows Embedded Standard 8. Kinect SDK Hello World: Hello peoples! Here is my “Hello World” Tutorial for the Microsoft Kinect using v1.8 SDKIn this tutorial we will be using Visual Studio (In my case 2015) to create a WPF Application using the Kinect to get you up and running with the SDK.

  • 2Installing the Microsoft Kinect SDK
  • 3Bringing the Kinect SDK in to Unity
    • 3.1Install the Unity Package
  • 4Using the Scripts
    • 4.1Setting up the Kinect_Prefab
    • 4.2Controlling your Character

What is the Kinect?

Installing the Microsoft Kinect SDK

Because this set of drivers is supported by Microsoft, setup is relatively easy - you just need to download the SDK installer package and follow the on-screen prompts.

Downloading and Installing the SDK

NOTE: Before installing, ensure that the Kinect is NOT connected to your machine.

  1. Download the SDK and Toolkit from http://www.microsoft.com/en-us/download/details.aspx?id=36996
  2. Run the installer and follow the on-screen prompts.
  3. Once the installer finishes, connect the Kinect to your computer.
  4. Feel free to run the Sample Skeletal Viewer to ensure the Kinect is functioning properly.

Integrating with Unity

PLEASE NOTE: Even if you don't have a physical Kinect attached to your machine, it is recommended to install the drivers on the machine you are developing on.

In order to utilize the drivers that you just installed, you will just need to use the provided scripts within your project.

Luckily, the Unity package provided on this very page will install all of the required assets for you. Check the next section for instructions how to both install and utilize it!

Bringing the Kinect SDK in to Unity

Install the Unity Package

The pre-built Unity package (Kinect Wrapper Package for Unity3D) includes all of the scripts required to start running your world.

To install, simply import the package, and you all of the required assets will appear in your project.

Kinect sdk 2.0

The following items are included within the package:

Scenes

  • KinectSample - This scene shows you how a skeleton is generated / tracked by placing spheres at each of the bones tracked by the Kinect, and how to use kinect to control your model. Use this to get a feel for what the Kinect is capable of. It also shows you how to prepare your GameObjects

Prefabs

  • Kinect_Prefab- This prefab has an empty node which contains all of the necessary scripts for your world to start using the Kinect - this does NOT include the controller, which you will need to use to actually control your models. See setup
  • KinectPointMan - This prefab shows how KinectPointController works.

Kinect

  • KinectModelControllerV2 - This is the script that you will attach to your model that you want to manipulate - you will drag each of the bones to be controlled into the appropriate slot, and determine which player controls which model.
  • KinectPointController - This script will place GameObjects that you define onto points that are tracked by the Kinect, generating a skeleton. The starting scene comes with an example of how this looks / should be done.
  • DisplayDepth - This script will get the depth image. Attach it to a game object with renderer. NOTE: In unity, you need to restart unity everytime after running your world, otherwise this script will not work (because of SDK issue).
  • DisplayColor - This script will get the RGB image. Attach it to a game object with renderer. NOTE: In unity, you need to restart unity everytime after running your world, otherwise this script will not work (because of SDK issue).
  • KinectRecorder - This script will record your movement and output playback files for the emulator.
  • KinectEmulator - This script will act as a virtual Kinect. It works with playback files. For now it only simulate the skeleton data.
  • KinectSensor - This script gets data from the physic Kinect.
  • DeviceOrEmulator - This script sets whether to use physic Kinect or the emulator.
  • SkeletonWrapper - This script grabs skeleton data.
Sdk
  • DepthWrapper - This script grabs depth image data.
  • KinectInterop - This script grabs data from Microsoft Kinect SDK.
  • Recordings/playbackDefault - This is the default playback file for emulator. Do NOT remove this file.

Models and Materials

  • rainbowMan
This is an example model that you can use for testing / learning how to use the wrapper. You can drag the KinectModelControllerV2 script onto this model and attach the bones and immediately begin testing.

Using the Scripts

After importing the asset package, you have the KinectExample to test and get familiar with the KinectPointController and KinectModelControllerV2. You can use this as your base scene, or you can drag the appropriate GameObjects into a new one.

Setting up the Kinect_Prefab

As previously mentioned, the Kinect_Prefab GameObject is required to have Unity talk with / use the Kinect. This empty object needs to exist somewhere in your scene. However, if you are switching scenes, it will persist because of the DontDestroyOnLoad call in the KinectWrapper script, so you don't need to instantiate a new Kinect_Prefab in each scene.

To get the best use out of the Kinect, you may need to tweak some settings within the Kinect_Prefab object itself - here is a breakdown of each option.

Device Or Emulator

  • Use Emulator - Check this to use emulator.
Kinect sdk 1.8

Kinect Sensor

NOTE: do NOT enable this script manually, it is controlled by DeviceOrEmulator.

Kinect sdk windows 7
  • Sensor Height - How high (in meters) off the ground is the sensor.
  • Kinect Center - This tells the Kinect where it should be looking for it's 0,0,0 point (relative to the ground directly under the sensor). The default works pretty well.
  • Look At - Tells the Kinect how to orient the camera using the motor control.

NOTE: The following values allow you to smooth the skeleton data. Usually the default values work fine. Do NOT change them unless you find magic numbers for you project.

  • Smoothing - Default value 0.5.
  • Correction - Default value 0.5.
  • Prediction - Default value 0.5.
  • Jitter Radius - Default value 0.05.
  • Max Deviation Radius - Default value 0.04.

Kinect Emulator

NOTE:

  1. do NOT enable this script manually, it is controlled by DeviceOrEmulator.
  2. If you are in the emulator mode, it will automatically play the playbackDefault file.
  3. Press F12 to play file of the path Input File. Press F12 again to switch back.
  • Input File - The file you recorded for the emulator. The default value is 'Assets/Kinect/Recordings/playback0'.
  • Sensor Height - How high (in meters) off the ground is the sensor.
  • Kinect Center - This tells the Kinect where it should be looking for it's 0,0,0 point (relative to the ground directly under the sensor). The default works pretty well.
  • Look At - Tells the Kinect how to orient the camera using the motor control.
Kinect Sdk

Kinect Recorder

Kinect Sdk Mac

NOTE:

  1. Press F10 to start recording, and press F10 again to stop.
  2. You can record multiple files in one run.
  3. You recording files will be created sequentially in the folder determined by Output File. The default value is 'Assets/Kinect/Recordings/playback', so the your recording files in one run will be named as 'playback0', 'playback1', etc.
  4. The recorder will overwrite existing files. Remember to save your file to another folder before you run your world again and do the recording; or you can change the value of Output File to save the new files to another place.
  • Output File - Where your recording files will go. The default value is 'Assets/Kinect/Recordings/playback'.

Controlling your Character

Simply put, to get your character moving, you need to first attach either the KinectModelControllerV2 to control the bones of the model, or KinectPointController to control a series of GameObjects.

NewModelController

To set this up, do the following:

  1. Drag the KinectModelControllerV2 script on to your model.
  2. Select the model, and find the Sw variable (it stands for Skeleton Wrapper). Drag your Kinect_Prefab from your CURRENT SCENE into this variable. This gives us a pointer to the main update script.
  3. Now, expand your model fully so that each bone is visible in the hierarchy.
  4. One by one, drag each bone you want to control onto the appropriate variable. Make sure to double-check the bones, as it's very easy to place it incorrectly.
  5. After all the bones are placed, set the player that should control this model. 0 is player one, and 1 is player two.
  6. Next, determine whether you want the entire skeleton to be affected by the Kinect (or just some parts) and set the appropriate mask. If you don't see one that you want, write your own!
  7. If you intend to animate your model while the player is controlling it, set 'animated' flag and determine how much blending between the animation and the Kinect should occur - this is a range from 0 to 1.

KinectPointController

To set this up, do the following:

Kinect Sdk 3

  1. Create an empty object to store each of the objects you want to control.
  2. Parent each of the objects into this empty node, and place them all at the same location.
  3. Attach the KinectPointController to the parent node.
  4. Select the parent node, and find the Sw variable (it stands for Skeleton Wrapper). Drag your Kinect_Prefab from your CURRENT SCENE into this variable. This gives us a pointer to the main update script.
  5. Drag each of the objects you want to be controlled into the appropriate variable.
  6. Set the player number (0 = player one, 1 = player two) and the Mask.

For an example of how this should work, check the KinectExample provided in the package.

Updates (Aug. 2013)

  1. Updated wrapper to Kinect SDK 1.7 in Unity 4.2.
  2. Added in Joint Orientation data from Kinect sensor.
  3. Added seat mode.
  4. Fixed the color and depth image display bug.

Kinect Sdk 1.6

Example Code and Drivers

Kinect SDK
Kinect Wrapper Package for Unity3D
Kinect Wrapper Example Project
Troubleshoot body tracking


Retrieved from 'https://wiki.etc.cmu.edu/index.php?title=Microsoft_Kinect_-_Microsoft_SDK&oldid=1056'