PoseCam
for Windows
&
PoseAI Studio

The Studio app visualizes the animations in real-time on an avatar, can record to FBX and use video files as source material.

The Studio app and PoseCam for Windows can stream to a game engine on the same PC (by using Localhost at 127.0.0.1) and in many cases avoid firewall or connectivity complications from using two separate devices. Streaming over networks via UDP is still supported.

The Windows version AI engine runs only on the CPUs, although the visualization app uses the GPU. Most desktop games and editors are GPU performance constrained, so relying on CPU-only will usually enhance the overall performance. The iOS app demonstrates our real-time motion capture engine running on mobile based GPUs.

Generally the cameras on iPhones are considerably better than most users' webcams, and better imaging helps the AI. For some applications, more granular control over shutter speed can make a big difference to the quality of the animation.

While our engine can also be embedded in Android apps, we do not plan on releasing our own app for Android at this point, as most creator use cases can use Windows or iOS.

A single low charge for 12 months of unlimited motion capture. No per-second, per-minute, per-video or processing charges. All for a fraction of what various alternative AI solutions charge.

No.

Our underlying AI engine was designed for real-time player input from a single camera. While the studio app offers the ability to record animations, it is not intended to be a replacement for a multi-camera, professional motion capture studio experience. The animations are recorded in real-time, without post-processing or cleaning, and exactly corresponds to what a live player would provide as input. We also do not calibrate the camera, the capture to the actor or the output to the avatar proportions.

Some developers have told us this tool could be useful for their game development pipeline, primarily to:

1. storyboard animations prior to recording (with actors) in a mocap studio, i.e. game prototyping and design stages

2. block out animations for professional animation artists to then modify, i.e. for import and editing in Maya/Autodesk/Blender

In the future we may consider adding post-processing features or multi-camera input to further streamline the creator process.

Research and Development - NO. You cannot use our software or AI engine for

• training AI models

• supervising AI models

• testing AI models

• reverse engineering AI models

• annotating images, videos or datasets

Commercial Games and Media - YES. You can use the animations for commercial purposes, such as generating animations for a game, cartoon, video, livestream or similar media, provided it is not for AI.

Interactive Experiences and Installations - YES with additional license. If you want to run our app to power an interactive experience, exhibit or installation, you just need an incremental public display license. We will aim to add that to the Microsoft Store in the future, but for now please email info@poseai.co.uk to arrange.

Animation Services, Artist Enhanced - YES. You can use our software commercially to generate animations for resale, for instance in .FBX form on an asset store, if you have subsequently significantly modified the animations with editing software. So using our software as a starting point for a skilled animator to create cool animations is fine.

Animation Services, Other - NO. Using our software to churn out animations for resale or providing a video-to-animation conversion surface is not licensed, nor will the final product be licensed in such cases.

Animations can easily be recorded, saved and exported into a common file format and used in other applications.

1. Open the PoseAI Studio app

2. Start the webcam or video animator

3. Select the rig format to use. This will determine the skeletal structure, bone names and base rotations for each joint

4. Click on the record button. When finished click on the stop recording button. The app will save a .fbx and .bvh file in the Exports folder of the app (accessible from the launcher app) with the body animation, and a .csv file with the facial blendshapes. The fbx and bvh file can be imported as normal into a game engine or editing software. The .csv file can be imported into the Unreal Engine using Epic's plugin, as if it were from Epic's iPhone app.

You can live stream animations from both PoseAI Studio and PoseCam, just as you could from our iPhone developer app.

Select the streaming option in settings of the Studio app or from PoseCam. Input the IP address and port number of the listening server. If running on the same computer, you may be able to use the LocalHost ip address by default (127.0.0.1), although if your apps are run with no-trust settings they may be siloed and local access blocked.

To connect with Unreal LiveLink, run our free marketplace plugin inside the Unreal Engine. To connect with Unity3d, run our scripts (available on Github) from inside the Unity3D editor. Or for a different stack, our animations are streamed in JSON packets - try our python script (available on Github) to see how to access the stream for further development.

Either app can run in the background on the same PC - PoseAI Studio offers more utility but uses more of a processing load, while PoseCam is lightweight but relies only on the openCV drivers and may not work as well with all webcams.

The app will default to the first camera it finds if multiple cameras are connected to the PC.

Currently in the PoseCam Studio app, if you select the OpenCV driver option (#13 in our quick start diagram found in the app's documents folder), you can change the camera by specifying the index in the small input box next to the OpenCv settings tickbox (#14 in our quick start diagram).

We may add the ability to select cameras to the base settings in the future.

Our AI generally does not need a very high resolution for body capture - often VGA suffices. However face and, particularly, hands need higher resolutions. If the subject is distant from the camera 1080p or 4k resolutions can be helpful. At seated-at-desktop range (i.e. 1-1.5m), usually 720p suffices.

When using the Studio app, while in non open-cv mode: start/stop the Webcam and then the dropdown box (#15 in our quick start pdf diagram) should populate with the detected camera resolutions, FPS and encoding types. Different encodings may have different latency and system resource usage. While in open-cv mode, the app will automatically request 1080p at 30fps from the camera, however the actual resolution and framerate will depend on device and driver implementation.

When using PoseCam app, you can request different resolutions. Note that some cameras do not play nice with opencv drivers, and will only get high frame rate when reduced to VGA resolution.

Our AI is very lightweight and on most gaming PCs can usually run at 30-60 fps while in the background. If the device has a decent specification, the most likely bottleneck is the drivers we use to connect to webcams - either as part of the Unreal Engine or via OpenCV backend. In some cases we may not be able to access the full capability of the webcam, such as the resolution or often the frame rate. Built-in laptop webcams are often the most problematic and usually lower quality than a peripheral webcam.

If the frame rate seems capped at a very low rate, like 5 fps, try reducing the resolution to VGA, as sometimes the camera drivers only responds to high frame rates using older formats. Also try both with and without the OpenCV option in our Studio app, as this option completely changes the underlying method used to access the webcam stream.

Some video formats are not readable by our included drivers. Generally base encoded mp4 or mov formatted videos work best. Other formats are more likely to crash and/or result in a low frame rate or high latency due to decoding (i.e. webm format).

For hand and face capture, high resolutions or closer shots are generally required, as otherwise the details are too low for optimal AI capture.

Many videos contain motion blur which can further mask the underlying motion.

Camera cuts, fade outs and changes of perspective will usually lead to a break in the animation, and can also cause the AI to lose track of the subject or switch to a different subject if multiple people are in the scene.

If you have feedback on our early access Windows release, please send to studio@pose-ai.com

If you have other issues, you can try support@pose-ai.com, or where appropriate flag an issue on our Github.

If you need to discuss licensing and embedding our AI engine in your own product, please write to info@pose-ai.com

We do not currently maintain a public Discord or Slack, and only support these channels for our corporate licensees.

Subscriptions and payments are managed by the Microsoft store, not by Pose AI. In order to cancel your subscription or trial, go to the Microsoft store.

1. Login directly to your account.microsoft.com account or access from the Microsoft Store app (click on your initials to access your profile and select Manage Account and Profile).

2. Select Services and Subscriptions from the menu bar on their website

3. Find the PoseCam entry and select Manage. You can cancel or change your billing information through this menu

We can not directly provide refunds, process billing issues or cancellations. Please use the Microsoft Store functionality for these issues and contact Microsoft directly if there is an issue with their storefront.

Scenario: The loading app is stuck on the step "Checking with Microsoft Store" or similar, and the buttons to launch the app are not activated.

Likely cause: The app needs to check the status with the Microsoft Store api via Microsoft servers, but your firewall may be blocking the app's access to the internet.

Potential Solution: Go into your firewall software and allow poseaistudioloader.exe (as well as a poseaicamera.exe and poseaistudio-win64-shipping.exe) access.

Please use our free trial period to test whether the Studio app works well with your hardware, as some of the below issues are out of our hands or unlikely to be resolved in the near-term.

1. If multiple cameras are attached, some modes do not provide switches for camera selection

2. Some webcams currently only provide a lower quality stream (either very low FPS or low resolution) compared to the built-in computer's camera apps or video conferencing solutions.

3. Using video as source material sometimes crashes the studio app.

4. The Studio app may crash at launch when using DirectX12 on some hardware, often laptops with system-on-a-chip such as Intel integrated GPUs. This is an Unreal Engine issue and something out of our control. We have provided an alternative launch button which loads the Studio app with DirectX11, which should sidestep this issue. The lighter PoseCam app does not use the Unreal engine and should not encounter this issue.