Using Vicon Pegasus

The following topics are covered:

• Solving and retargeting for media applications (Shogun or Blade)
• Solving from rigid bodies (Tracker) -- Pegasus Advanced only
• Solving virtual cameras (Solving from props)
• Tuning the solve
• Saving and using templates
• Running multiple instances of Pegasus and run-time solvers
• Unreal plug-in integration
• Unity plug-in integration

Video tutorials for Vicon Pegasus are also available. For information, contact Vicon Support.

Solving and retargeting for media applications (Shogun or Blade)

To solve and retarget for Shogun or Blade:

1. On the File menu, click New project to start a new project.
2. In the Server settings dialog box, enter the server settings (IP number, port number, and source server for Shogun or Blade).
   

3. On the toolbar, click Import characters from Vicon datastream . This loads a skeleton in the scene. Click the button as many times as necessary to load the correct pose for matching.

   

4. On the File menu, click Import FBX to import the target avatar from an FBX file.
   
5. To match the source mocap data to the target avatar and position the avatar to the mocap data:
   
   1. Select 1 bone from the source data (the skeleton you imported in step 3) and CTRL+select 1 bone from the target avatar you imported in step 4.

   2. On the toolbar, click the Associate Characters button .

      You can also match the source to the target without scaling or using pre-stored templates. For more information, see Associate Characters in the Vicon Pegasus toolbar table.

      

6. On the toolbar, click the Rotate tool .
7. Adjust the match pose by manipulating the local rotation of bones of either the source data or the target avatar.
   

8. Select the foot of the source avatar, CTRL+select the foot of the target avatar and on the toolbar, click the Link bone to task button  to create IKinema Position and Orientation Tasks (IK constraints).

   This will maintain the feet position and orientation in world space during solving.
   

   If required, you can also:

   • Maintain zero offset between the source and target feet bones by selecting Snap feet to source.
   • Increase the importance of the IK constraints by increasing the Weight for the selected task on the IKinema tab.
     

   Note

   When you manually associate target joints with source objects (joints or rigid bodies) with Pegasus 1.2, you no longer need to ensure that they are selected in the correct order.

9. On the toolbar, click the Start/Stop Pegasus RunTime button  to start the Real Time IKinema solver.

   

10. If necessary, stop the RealTime pipeline and adjust match pose or solvers.

Solving from rigid bodies (Tracker) – Pegasus Advanced only

The following procedure applies only to users who have an Advanced Pegasus license.

To solve from Tracker rigid bodies:

1. On the File menu, click New project to start a new project.
2.  In the Server settings dialog box, enter the server settings (IP number, port number and Tracker input).
   

3. On the toolbar, click Import characters from Vicon datastream  to import the Tracker rigid bodies.

   

4. To group the rigid bodies into a subject, select the bodies, then on the Edit menu, click Group selection and in the dialog box, enter the name of the subject.
   
   The new subject appears in the viewport.
   
5. On the File menu, click Import FBX to import the target avatar from an FBX file. This could be Jack, V6, or any other subject.
   
6. Select one rigid body from the grouped subject and CTRL+select the imported avatar.
7. On the toolbar, click the Associate Characters button  and then click Scale and move target to rigid bodies. This matches the avatar to the imported rigid bodies.

8. If necessary, manually adjust the match pose using the Rotate tool .

   

9. When the avatar is matched to the rigid bodies, start associating the bodies to the skeleton. To do this:
   1. Select the source (rigid body) then CTRL+select a bone and on the toolbar, click the Link bone to task button  to create an IKinema Position and Orientation Task that will drive the body.
   2. Continue associating the bodies to the skeleton. Typically, 9–11 rigid bodies are required to set up a full body solve.
      

   Note

   When you manually associate target joints with source objects (joints or rigid bodies) with Pegasus 1.2, you no longer need to ensure that they are selected in the correct order.

10. On the toolbar, click the Start/Stop Pegasus RunTime button  to start the Real Time IKinema solver.

    Data starts streaming to the run-time pipeline and UI.
    

Solving virtual cameras (Solving from props)

1. Load a scene with a virtual camera or props.
   
2. Load an FBX model of a virtual camera.
   
3. Select a bone from the virtual camera (source) and CTRL+Select a bone from the FBX model of a camera.

4. On the toolbar, click the Associate Characters button  and then click Associate Character without Scaling.

   

5. Using the Group node on the FBX model of a camera and the Rotate tool , adjust the direction of the camera to match the Shogun or Blade virtual camera.

   

The same procedure applies for solving any other prop.

Tuning the solve

By default all the necessary tuning is applied to the solve. In some situations, you may need to perform one or more of the following additional tuning steps:

• Link bones manually: Ensure the automatic match has linked all bones in media solving. Link bones manually if necessary.
• Exclude hierarchies from the solve, for example, head rigs, fingers, auxiliary bones, etc.
• Disable roll bones from the solve.
• Increase stiffness for bones that do not have direct drive. For Shogun or Blade, increase the stiffness for bones that do not have a retargeting source, for example, spine bones. The stiffness varies from 0–1. For spine, typical values are 0.3–0.7.
• For Tracker solving, increase the stiffness of clavicle bones to 0.85–0.99 to add some rigidity since in most applications these are not driven by rigid bodies.
• Increase the Weight on some specific tasks, for example, feet to 5–10.
• Add additional task constraints in Shogun or Blade solving applications, for example, on the chest, to bring the upper body closer to the reference solve.

Saving and using templates

Pegasus enables you to save custom templates for later use, which can be useful for re-using manually added links.

During the association of avatars, the retargeting links are automatically created, based on naming conventions. If for some reason the link cannot be generated automatically, you can link the source to the target manually and then save the link in a template.

To save manual links to a template:

1. Manually link a source bone to a target bone, by selecting the source and target and then click the Link to bone manually button , or Link to task button  on the toolbar.
2. To save this link to a template, on the Edit menu, click Save template and in the dialog box, enter a name for the template.
   
   The template is saved as a matching file under a template folder in the main Pegasus folder.

After you have saved a template, you can use it to automatically associate source to target bones. To do this:

1. Select first the source then CTRL+select the target.
2. On the Edit menu, click Apply template and in the dialog box, select the required template.
   

Running multiple instances of Pegasus and run-time solvers

If necessary you can run multiple instances of Pegasus and run-time solvers.

To do this:

• In the Output settings dialog box, specify a different DataStream Output Port for each instance of Pegasus, for example, 802, 806, 810, etc.
  You can then run Pegasus in detached mode and close the user interfaces. This creates multiple real-time instances that stream solved data to your game engine and application.

Unreal plug-in integration

Before you can use the Unreal plug-in, ensure you have installed the required version of the Unreal Engine and have copied the plug-in files to the appropriate folder. You can then use the plug-in. For more information, see:

• Installing the Unreal plug-in for Vicon Pegasus
• Using the Unreal plug-in for Vicon Pegasus
• Using a virtual camera with the Unreal Engine

Installing the Unreal plug-in for Vicon Pegasus

To install the Unreal plug-in for Vicon Pegasus:

1. Install the Unreal Engine, version 4.5.0. (If you are using an older version of the Unreal Engine, contact Vicon Support for more information.) By default, the Unreal Engine is installed in C:\Program Files\Unreal\Unreal Engine\4.5.
2. Install Vicon Pegasus. When you install Pegasus, the Unreal plug-in files are copied to a a folder called UEPlugin, under the location of the Pegasus installation (by default C:\Program Files\Vicon\Pegasus1.1.0\UEPlugin).
3. Install the plug-in by copying all the files from the Pegasus UEPlugin folder, by default:
   C:\Program Files\Vicon\Pegasus1.1.0\UEPlugin
   To the Unreal Engine installation folder, by default:
   C:\Program Files\Unreal\Unreal Engine\4.5

Using the Unreal plug-in for Vicon Pegasus

To use the Unreal plug-in with Vicon Pegasus:

1. Launch the Unreal Engine.
2. Create a new project with first person shooter blueprint.
3. In the editor, on the Edit menu, click Plugins.
4. In the dialog box:
   1. Select Retargeting.
   2. To enable the Ikinema DataStream plugin, select the Enable check box and then click Restart Now.
5. When the editor has restarted, right-click in the folder structure in the content browser and create a new folder.
   
6. To the right of where you added your folder (in the area labeled Drop files here or right click to create content), right-click and then click Import to.
   1. Browse to the location of the avatar used in Pegasus and confirm.
   2. Ensure Skeletal Mesh is highlighted and click Import All.
   3. Dismiss the log message.
7. In same space, right-click, point to Animation and click Animation Blueprint.
8. In the dialog box, select the avatar from the list in the bottom section and click OK.
9. Give the new item an appropriate name.
10. Double-click the item you created and in the dialog box:
    1. Click on the text Result in the final animation pose object and drag into blank space.
       
    2. From the list, select Ikinema and Ikinema Solved Skeleton.
       A line is created from the Final Animation Pose item to the Ikinema Solved Skeleton item.
       
    3. If you are using only one PC and default settings, enter the following:
       
       • Server Name – localhost
       • Port Number – 802

       • Subject Name – The name of the character in Subject and Solvers pane.
         

         Important

         Subject name refers to the character name in brackets next to the FBX name and not that of the avatar/fbx imported into Vicon Pegasus.

11. Click compile.
    You should now see circles moving from the Ikinema Solved Skeleton item to the Final Animation Pose.
    

When you have completed the above steps, you should see your animation playing back in the preview pane. You can view the animation in the scene by dragging and dropping the animation into the scene. You can also enter a 1st person preview mode.

Using a virtual camera with the Unreal Engine

To add a virtual camera:

1. Launch the Unreal Engine.
2. Create a new project with first person shooter blueprint.
3. In the editor, on the Window menu, click Plugins.
4. In the dialog box:
   1. Select Retargeting.
   2. To enable the Ikinema DataStream plugin, select the Enable check box and then click Restart Now.
5. When the editor has restarted, right-click in the folder structure in the content browser and create a new folder.
6. Within the same folder create two more folders – one for subject and one for prop/vcam.
7. Select one of the two folders and to the right of where you added your folder (where it says Drop files here or right click to create content), right-click and then click Import to.
   1. Browse to the location of the avatar used in Pegasus and confirm.
   2. Expand the Advanced options and select Import Materials and Import Textures.
   3. Ensure Skeletal Mesh is highlighted and then click Import All.
   4. Dismiss the log message
8. Repeat the process described in step 7 in the second folder for the prop/vcam mesh/fbx.
9. In the avatar folder, right-click in the space where you imported your fbx, point to Animation and then click Animation Blueprint. In the dialog box:
   1. Select the avatar from the list in the bottom section and click OK.
   2. Give the new item an appropriate name.
10. Double-click the item you created.
11. In the new dialog box:
    1. Click on the text Result in the final animation pose object and drag into blank space.
    2. From the list, select Ikinema and Ikinema Solved Skeleton.
    A line is created from the Final Animation Pose item to the Ikinema Solved skeleton item.
12. If you are only using 1 PC and default settings enter the following:
    
    • Server Name – localhost
    • Port Number – 802

    • Subject Name – The name of the character in the Subject and Solvers pane.
      

      Important

      Subject name refers to the character name in brackets next to the FBX name and not that of the avatar/fbx imported into Vicon Pegasus.

13. Click compile.
    Circles move from the Ikinema Solved Skeleton item to the Final Animation Pose and the animation playing back in the preview pane.
    

14. Repeat steps 5–7 for the prop/vcam.
15. Drag the animation blueprint from each folder into the scene.
16. In the Modes pane (top left by default) type camera into the search field and drag Camera into the 3D workspace.
    
17. In the scene Outliner pane locate the camera you just dragged into the scene. Click and drag this onto the prop/vcam to attach it. Pick a joint and note it down.
    The Outliner appears similar to the following example:
    
18. To simulate and reselect the camera module, click Play. The camera module should move around in a preview pane.
    

Unity plug-in integration

Before you can use the Unity plug-in, ensure you have installed the Unity Plugin Sample and have copied the plug-in files to the appropriate folder. You can then integrate the supplied sample code into your own Unity application. For more information, see:

• Installing the Unity plug-in files
• Using the Unity plug-in

Installing the Unity plug-in files

The Unity plug-in is not installed by default, so make sure you select the option when you install Vicon Pegasus.

By default, the Unity Plugin Sample is installed to the \UnityPlugin directory under the Pegasus installation.

Because the solution is inside the Program Files folder, you will need Administrator permissions to write anything to the default folder. You can either:

• Copy the contents of \UnityPlugin to a separate folder before opening and building the project; or
• Make sure you are running with Administrator permissions when you open the solution.

Using the Unity plug-in

To use the Unity plug-in:

1. Ensure you have installed the Unity plug-in as described in Installing the Unity plug-in files above.
2. Open the PegasusUnitySample.sln file in MonoDevelop. If the references to the UnityEngine are displayed as broken, edit Assembly-CSharp.csproj to point it at your Unity installation.
3. Build the solution (F8) and run it (F5).
   The Unity launcher starts and asks for a project.
4. Select Open Other and point it at the UnityPlugin directory (or, if you moved the solution file, its parent directory).
   A blank scene opens.
5. In Unity, drag the Program.cs asset on to the camera in the scene.
6. Start Pegasus streaming data, then click the Play button at the top of Unity to start the stream listener.
   The console window displays text output, which indicates that data is being received from Pegasus.