Note: The advanced sample project available on our Github page demonstrates our recommended method for connecting the app to Unreal for gameplay. Our improved plugin includes helper nodes which manage the LiveLink source connections, both for the character and animation blueprints.
- 01
Install the PoseAILiveLink plugin from the Unreal Marketplace or copy the folder into the [Your Project Name]/Plugins/ folder. Make sure the plugin is enabled from within the editor (Edit/Plugins). This may require restarting the project.
Go to project settings->plugins->UDP Messaging. Make sure enable UDP transport is checked. If this is not available please enable Epic's UDP Messaging plugin from Edit->Plugins as LiveLink is unlikely to work without it.
- 02
On MacOS the marketplace build of the Pose AI plugin may not work when installed as usual to the Engine/Plugins/Marketplace/ folder, and when enabled can prevent the Engine from opening a project.
A simple workaround is copying the PoseAILiveLink plugin folder instead to <yourprojectname>/Plugins/. Once this is done uninstall the plugin from the Engine and it should work as intended. Alternatively rebuilding from source may work as well.</yourprojectname>
We are trying to resolve this with Epic as we understand this may be a known issue within the engine.
- 03
From the Unreal Engine menu select Window/LiveLink. From the LiveLink popup window add a Source->Pose AI App.
Chose among the various configurations depending on use case. Make sure you select a Rig which matches your target character skeleton, and select the mode depending on where you will position your phone (Room if you have space for full body capture, Desktop if you are only focusing on upper body and hand movements).
Click OK when ready. PoseAI Mobile should be added to the first panel of LiveLink.
- 04
By default LiveLink buffers several frames of motion capture data. This can add latency to the input from Pose Camera. On a local network and with Pose Camera's in-app smoothing, we recommend reducing the buffer to 1 frame, and only if necessary increase the buffer.
Click on the Pose AI mobile source in the top LiveLink panel and in the right details, under Buffer-Settings reduce "Max Number of Frame to Buffered" to 1.
- 05
Open the PoseCamera app on your phone. Go to settings (gear icon) and input the IP address of the computer running the Unreal Engine and the port number selected when configuring LiveLink (the first panel of LiveLink will display the first local host IP address detected but if you have multiple adapters this may not be correct). Make sure your phone and the computer are on the same local network.
Close settings on your phone. The app should show the IP address you entered at the bottom of the main screen. Tap to connect. When successful the phone app should display "Unreal LiveLink" in place of the IP address.
Unsuccessful? Try our troubleshooting page for tips resolving common connection issues
- 06
Once connected the phone app should offer "Tap to Stream". Tap and the phone should switch to camera preview mode.
Within Unreal you should see your phone's name in the bottom LiveLink panel and will show green when it is streaming detected poses.
- 07
Place your phone appropriately to capture your target space with an unobstructed view. For example, in Desktop mode you might place your phone underneath your monitor or in Room mode on your AV console. You can toggle between front/back camera. You can even aim your phone camera at media playing on another device.
Make sure the camera can see most of the target. For Room or Portrait ensure the full body, head to feet, is visible. For Desktop mode ensure the camera can see the head to pelvis and ideally both elbows. The segmented human icon overlaying camera preview turns blue when a subject is visible and shows which limbs are currently poseable.
Read our troubleshooting page, FAQ or our Unreal Example Walkthrough for more help