Shogun Post's full retargeting pipeline enables you to create and solve motion capture data onto any custom FBX/USD biped.
You can use the resulting VSR file in Shogun Live and stream it to a game engine, or solve and export from Shogun Post for use in a CG app.
You can use either Shogun Post or the Vicon Retarget app for retargeting:
- Use Shogun Post for retargeting instead of Vicon Retarget if you want script the process, need to align the source skeleton to the target skeleton or change the bone lengths of your target skeleton to better match the source.
- Use the Vicon Retarget app if you need Joint Symmetry or Target Symmetry.
The current retargeting implementation is Phase 1 of a longer planned roadmap. Use the retargeting workflow as a first step in the retargeting process, and as a quick and easy way to get your custom characters driven by Vicon mocap.
Set up a retarget
To set up a retarget in Shogun Post, see the following information.
- Source skeleton: Solving skeleton
- Target skeleton: Retargeting skeleton (game skeleton)
- Map pose: The pose that is used for mapping (creating constraints between the source (solving) and target (retargeting) skeletons and setting the offsets). For more information, see Map pose and best rig practices in Getting more from Vicon Shogun.
- Base pose: The pose of the skeleton when all keys are deleted and the rotation channel for all bones is set to zeros. It's defined by the pre-rotation, which is known as ‘joint orient’ in Autodesk® Maya®.
Create an optimal target skeleton
The following tips help you to ensure that your FBX, USD or VSR file works well as a target.
Use pre-rotation, not keys
It's best to define the base pose of a skeleton using pre-rotation, not keys.
If keys put the skeleton into a T-pose, A-pose, or another suitable pose, and pre-rotations don't exist, you can still use the skeleton, but you'll need to know what the base pose looks like and use the map pose to both retain the pose used for mapping and to get best results. See About map mode and, if your skeleton uses keys rather than pre-rotations, see also Map pose and best rig practices in Getting more from Vicon Shogun.
Pay attention to degrees of freedom (DoFs)
Most skeletons have Degrees of Freedom (DoFs), and generally, skeletons work best when they have them.
- FBX files: After you import an FBX, check that the DoFs are set correctly before setting up retargeting.
- USD files: The USD format doesn't currently support DoFs. If you want to use DoFs with a USD skeleton, set them in Post after importing the USD skeleton.
Be aware of dummy bones
End joints that are dummy bones (bones with zero DoFs) are ignored by retargeting in that any keys they may have aren't retained and a map pose isn't stored for them.
- For extremities like the ends of hands, toes, and fingers, this is fine because the end bones must be zero DoF dummy bones with no keys.
- For parts of a skeleton that define things such as face, hair, and clothes, this can be problematic if these bones don't use pre-rotation to define their base pose. If you use keys to define a pose that is required for the skin to look right, note that these keys are not stored in the VSR/map pose because all of these bones are treated as dummy bones. If they aren't dummy bones already, change them to dummy bones because they have no data and setting them to dummy bones enables retargeting to ignore them and therefore to run faster.
To convert unused bones to dummy bones in Shogun Post, either:
- If your bones have pre-rotation values that you want to keep, note their values. In the Subject Setup panel, on the Retargeting tab, click the Prep Unused Bones button. This ensures all unused bones have zero DoFs and moves their rotations from channels to pre-rotations. You can enter the pre-rotation values that you noted to manually reset them.
- Manually set the DoFs to off. To do this, in the table on the Retargeting tab, select the bones you want to be dummy bones and clear the Active check box.
Know what's above the root bone
Sometimes, skeletons in CG apps like Autodesk® Maya® or Autodesk® MotionBuilder® contain one or many bones, locators, groups, etc, above the root. Generally Vicon Shogun doesn't support this. Aim to have no extra nodes between the Retargeting node and the root bone in Shogun Post. Dummy bones can exist above the root. These must have zero DoFs and the root must have six DoFs. If this isn't already the case on import, in the Target Root field, specify the correct root, which switches DoFs.
In Shogun Post, on the Retargeting tab, you can select or clear the Preserve dummies above root check box, which controls whether the dummy bones above the root are exported in the VSR.
If any objects above the root have non-zero transforms, they can be successfully imported in FBX format, but you'll probably need to modify the hierarchy in Post to set them aside during setup. If you need them to exist in the exported FBX because it will be merged into the original file, you must restore the hierarchy to its original form after you finish retargeting. You can create export scripts to do this via batching.
Know how to use scaling
Scale (values in the Scale channel) is not supported on bones or anything above them.
Retargeting enables you to scale each joint so that it better matches the source skeleton. You can revert this back to the original scale on export, if desired (see Export FBX).
Import the target skeleton into Shogun Post
You can import the target skeleton as an FBX or USD file, or VSR, if a setup already exists. Ideally the imported file will contain just the skeleton and skin and as little else as possible.
To import the target skeleton:
- In Post, open the relevant .MCP file and ensure that the required solving skeleton is selected in the Current Subject field.
- To check that no target skeleton has been imported, open the Subject Management panel.
The Retargeting column displays a red circle, indicating that no retargeting setup is present.
For more information on the Subject Management panel, watch Vicon Shogun 1.3 Post Tutorial - Subject Management on YouTube.
- Do either of the following:
- In the Subject Setup panel, select the Retargeting tab, click the Load button and select the required target.
- Drag the target skeleton file into the Shogun Post view pane.
When you are prompted, choose Skeleton.
- In the Subject Setup panel, select the Retargeting tab, click the Load button and select the required target.
The target skeleton is imported into the current subject hierarchy with the topmost bone of the target skeleton parented to a Retargeting node.
When you import an FBX, it sometimes isn't added under the Retargeting node. In this case, manually parent the skeleton to the node (select the required nodes and on the Objects menu, click Parent).
Prepare the skeleton before posing
- Hide any unnecessary joints. To do this, select the bone, and in the Attributes panel, clear the Showing box.
- On the Retargeting tab, click Map Mode (see About map mode) to enable this mode.
This creates a separate clip that you use to pose the target skeleton relative to the source skeleton.
- In the Target Root field, specify which bone is the root. This ensures the root has six DoFs and any bones above it have zero DoFs.
- Confirm the target skeleton's DoFs were imported or set them as required.
- If the source and target skeletons don’t have roots in the same position or have differing orientation, on the Retargeting tab, click Align Skeletons.
Shogun Post tries to put the skeletons into the same positions and root pose.
You can more closely align the skeletons using the Translate and Rotate manipulators (not the Special manipulator).
Be sure to move the root, not any dummy bones above it.
- If you need to globally scale the target skeleton to be the same size as the source skeleton, on the Retargeting tab, change the Target Scale value. Ensure that the ankles and clavicles in the target and source skeletons match.
About map mode
The process of creating the constraints between the source (solving) and target (retargeting) skeleton and setting the offsets is called mapping. Before mapping, both skeletons must be in the same pose. This involves posing the target skeleton to match the source (which is its base pose) and you do this in map mode. First you pose the target, then you create the constraints. At the time a constraint is created, its offset is calculated. The offset is the difference in pose between the source and target.
If you need to tweak the setup, or just check how it was set up, you can return to map mode. When making any adjustments, the target must be in the same pose it was when it was mapped. The map pose makes this possible. If constraints exist, implying a map pose is likely to exist, entering map mode automatically places the target skeleton in the map pose. If needed, you can also use the Go to Map Pose button (on the Retargeting tab of the Subject Setup panel) to set the retarget skeleton's pose to the map pose.
After you've finished posing, to define the map pose, click Set Map Pose.
In addition to restoring the pose you used for mapping, the map pose is also useful for retargeting. Retargeting is faster and easier if the map pose is set because it uses the map pose as a starting point instead of the base pose, which, for some skeletons, may differ widely from the map pose.
Pose the skeleton
- Make sure that the target skeleton is keyed in the same pose as the source skeleton.
- Start with the root. Make sure that you’ve selected the actual root and not any dummy bones above it. Using the Translate and Rotate manipulators (not the Special manipulator), move it to the same place as the source skeleton.
- Use the Rotate manipulator (or enter values in the Channels panel) to rotate all the target bones to the same pose as the source skeleton.
Note you can also rotate the source skeleton to match the target skeleton or a mix of the two.
- If you need to change bone length to make the target skeleton exactly the same proportion as the source (assuming your pipeline both allows this), use the Special manipulator. Note that you must remove the GlobalScale retargeting parameter. This removes the ability to perform global scaling and unscaling (using the Unscale button), so ensure you've scaled your target skeleton first.
- When you have finished posing the skeleton, in the Subject Setup panel, on the Retargeting tab, click the Set Map Pose button to store the matching pose. If you later modify the pose, remember to click this button again so that the map pose is updated.
- In the lists on the right of the Retargeting tab (or anywhere else in Shogun Post), select a matching source and target bone, then click Add Position or Add Rotation to create a constraint between them. The order of selection does not matter.
- As a starting point for the constraints, add a position on hands, feet and hips; and rotation on all joints.
If your target skeleton has more joints than the Vicon source skeleton, you can constrain multiple target joints to the same Vicon source joint.Tip
You can use HSL scripting to speed up creating constraints. For information and examples, see attach in HSL scripting with Vicon Shogun.
- You can alter multiple constraint weights in the table on the Retargeting tab.
You can also select or clear Active to turn constraints on or off.
- To check targets, ensure that the View Filters option for Constraints (for Retarget) is selected.
- Set weights for all rotations. The recommended value for Rotation weights is 200. To set multiple rotations to 200, on the Retargeting tab, drag to select the required rows in the table and then in the Weight column, set the value of one of them to 200.
All the selected rotation weights are updated to 200.
- To save the VSR mapping file, at the top right of the Retargeting tab, click the Save button.
By default, VSR files are saved to C:\Users\Public\Documents\Vicon\Retargets.
Test the retarget setup
You can test the retarget setup in Shogun Post or in Shogun Live.
To test the retarget setup in Post:
- When you have finished setup, click Map Mode again to turn it off.
You are returned to the motion in the file, ideally a ROM.
- In the Subject Setup toolbar, click the Retarget Play Range button.
- Review the retarget.
To test the retarget setup in Live:
- Load the VSR. To do this, on the Processing tab, in the Retarget section supply the required names:
- Filenames. Enter or browse to the required VSR files. You can select or enter multiple files, separated with a comma.
- Names Enter the subject name(s). You can enter multiple subject names, separated with a comma. Ensure that the order of multiple names matches that of the VSR file names.
- In the General section, ensure that the Processing Output Level is set to Retarget.
Your FBX is displayed and is driven by the source skeleton. The retarget skeleton is recorded as part of the MCP capture.
If after testing, you find that further changes are needed, return to map mode and modify the setup (see Modify a retarget).
Modify a retarget
To check and modify your retargeting:
- In Shogun Post, make sure you have turned off Map Mode and then run Retarget from the Processing menu.
- Assess the results.
When checking the solve, if you need to reduce arm pull, try adding position targets on the clavicles. If the hands need to match more closely, consider zeroing the hand position offsets.
To see any changes, re-run Retarget from the Processing menu.
- Pay particular attention to the finger solve. Make sure the hands look correct.
- If necessary, update the joint placement to get a better fit.
For more information, see:
Vicon Shogun 1.3 Post Tutorial - Finger Solve Adjustments on YouTube.
- When you have finished updating the joint placement, in the Subject Setup panel, on the Retargeting tab, click Update Offsets to use the new joint positions.
- Re-solve the range and check the updated target.
- Save the final VDF and export the updated VSR.
Export the retarget setup
When you're happy with the setup, or if you'd like to see how it works on other motions, export the file, which contains the retarget skeleton, the retargeting constraints, and the map pose.
When you export in VSR format, you can:
- Load the VSR into other takes that only have a solving skeleton and then run retargeting.
- Load the VSR into Shogun Live to use it in real time.
- To export in FBX format, select the Retargeting node, then right-click and click Select Branch.
This selects the target skeleton, so that you can choose to export Selected objects only in the next step.
- To choose what is exported and whether to reset scaling on export, open the Preferences dialog box (on the General menu, click Preferences), click the File Export tab and in the File Format box, select FBX Motion.
- If you want to reset scaling on export, ensure that the Unscale retargeting skeletons option is selected.
See also Export from Shogun Post.
You can now test the exported FBX in your CG app.