All Collections
General FAQs
Your First Playtest
Understanding AI-Powered Analysis
Understanding AI-Powered Analysis

This article explains what you can expect when AI-Powered Analysis is enabled for your playtest and how it works

Rohan Joseph avatar
Written by Rohan Joseph
Updated over a week ago

Our brand new AI-Powered Analysis feature allows you to save time, get deep insights, and make better decisions for your playtest videos. It leverages AI to generate annotations and summaries for your playtest videos, giving you more time to navigate through and analyze your playtest.


How do I enable AI-Powered analysis for a playtest?

If your organization has access to AI-Powered Analysis, it is enabled for new playtests by default. If you would like to disable AI-Powered Analysis for a specific playtest, select the No, thanks button at the bottom of the AI-Powered Analysis section when ordering your playtest.

Which playtest types support AI-Powered Analysis?

AI-Powered Analysis is available for Single-session playtests, Multi-session playtests, Longitudinal studies, Concept tests, Player Interviews, and Live Playtests. It is not available for Survey only studies.

Where do I see the results of the AI-Powered Analysis?

When AI-Powered Analysis has successfully run for a playtest video, you will see annotations and a summary added to the video.

AI-Powered Annotations:

The AI-Powered annotations are visible in the Annotations tab of the video with the author set to AI. You can use the annotations Filter to show or hide the AI-Powered annotations on each video by selecting the AI Annotations toggle button.

The hashtags added in annotations by the AI-Powered Analysis will also be visible on the playtest overview page.

AI-Powered Summaries:

In the Summary tab of the playtest video, you will see an AI-Powered Summary for the video below the Internal Comments section. You can choose to show/hide the AI-Powered Summary for the video using the toggle button above the AI-Powered summary.

This summary will also be visible on the playtest overview page on the video blurb and will be tagged in gray as AI-Powered Summary.

How does AI-Powered Analysis work?

We are currently using the GPT-4 API (from OpenAI) to analyze playtest video transcripts to generate annotations and summaries. This feature is aimed at helping you analyze playtest videos faster by highlighting where you might uncover insights.

Think of it as a map to your playtest results, helping you navigate each video from the get-go.

We use AI to highlight moments where the player makes comments related to five categories:

  • Usability

  • Game design

  • Enjoyment

  • Monetization

  • Frustration.

Additionally, this feature uses AI to summarize all observations made into a short paragraph that sums up the experience each player had during your playtest. We then convert the results into annotations and summaries and present them along your playtest videos.

What playtest data is used to generate the AI-Powered Analysis?

The AI-Powered Analysis is based on the transcript of the playtest video. We also share the game name that was added while creating the playtest with the AI to give context on the playtest.

No other information regarding the player, game studio, or game is shared with the AI, and no personal data is shared alongside the transcript.
​
We currently use OpenAI's GPT-4 model and have not opted-in for playtest data to be used for training GPT-4 (OpenAI's data usage policy). No personal data of players or our customers is shared with OpenAI.
​
As the field of AI is rapidly developing, we are constantly evaluating new AI systems to improve AI-Powered Analysis. If we switch to another solution, we will ensure that similar guarantees on data use are in place.

Will AI-Powered Analysis work for Bring-Your-Own-Players?

Yes, AI-Powered Analysis is available for playtests that use our Bring-Your-Own-Players (BYOP) feature.

If your BYOP playtest is happening in a language other than English, please contact us at hello@playtestcloud.com so we can make some adjustments to the our internal configurations.

Will AI-Powered Analysis require changes to the Terms of Service or DPA?

No, using AI-Powered Analysis is possible under the existing agreements that we operate under with our customers. Your data continues to be handled securely and confidentially, in line with our agreements. Should there be an exception due to custom legal agreements in place between PlaytestCloud and your company, we will proactively reach out to you and you will have to opt-in to the use of AI-Powered Analysis.

Regarding Bring-Your-Own-Players and PlaytestCloud's status as a data processor in our relationship with our clients: The AI provider we use (currently OpenAI) does not receive any personal data from us during the processing of the videos, meaning they are not a sub-processor under our DPA.

What if the AI-Powered Analysis doesn't make sense?

There is a certain amount of uncertainty and randomness involved when working with any AI. On the off-chance the results of the analysis do not make sense or lack context, we have two options:

  1. Re-run the AI-Powered Analysis: contact us at hello@playtestcloud.com to initiate the re-run.

  2. Share feedback on why the analysis isn't helpful: We are always interested in improving the instructions we use for AI-Powered Analysis. We can look into making a general change to our instructions or making a modification for your playtest and re-running the analysis.

Can I run AI-Powered Analysis on older playtests?

Yes, this is possible with a special request. Reach out to us at hello@playtestcloud.com and let us know which playtest you would like to have analyzed.

Can I customize the analysis?

As a special request, you can share your research goals and objectives with us when placing an order. To do so, please include your research goals and objectives in the 'Special Request' section of your order form and mention that these should be used for the AI analysis. We can explore tweaking portions of our instructions to the AI to provide results that closely match what you are looking to find insights on.

How do I enable AI-Powered Analysis for my organization?

AI-Powered Analysis is available for all current subscription plans, the Starter Plan, and one-off projects. It is not available to Booster Packages, Professional Plans (2019), and Legacy Playtesting Credits users.

If you're using one of the plans that do not have access and would like to gain access, please reach out to us at hello@playtestcloud.com.

Important: If your account should have access to AI-Powered Analysis but you do not see it in your account, please check with your organization's admin.

Did this answer your question?