First Findings helps you save time, get deep insights, and make better decisions for your playtest results. It leverages AI to generate annotations and summaries for your playtest videos along with a playtest report combining all its findings, giving you more time to navigate through and analyze your playtest.
How do I enable and set up First Findings AI-Powered analysis for a playtest?
If your organization has access to First Findings AI-Powered analysis, it is enabled for new playtests by default. If you would like to disable it for a specific playtest, select No, thanks in the section when ordering your playtest.
You can configure the analysis while setting up your playtest:
Pick the categories used for the analysis: You can select up to 6 categories for the analysis. By default, we pre-select 5 categories: Usability & Understanding, Game design & Mechanics, Enjoyment, Frustration and Monetization, but you can pick from any of the 13 categories available.
Describe your game: Add any details about your game or the mechanics in the game to help the AI process the playtest data better. For example, adding details about the expected game difficulty level will help the AI distinguish between genuine player frustration and the player struggling to cross a difficult level.
Add your research questions: Sharing your research questions will help the AI structure the report to help you get to your answers sooner. It will prioritize your question over other insights.
Which playtest types support AI analysis?
First Findings is available for Single-session playtests, Multi-session playtests, Longitudinal studies, Concept tests, Player Interviews, and Moderated Playtests. It is not available for Survey only studies.
It is also not available if you have opted for no audio to be recorded by selecting No under Should players think aloud while playing?
Where do I see the results of First Findings AI-Powered analysis?
AI Annotations:
The AI annotations are visible in the Annotations tab of the video with the author set to AI. You can use the annotations Filter to show or hide the AI annotations on each video by selecting the AI Annotations toggle button.
AI Summary:
In the Summary tab of the playtest video, you will see an AI Summary for the video below the Internal Comments section. You can choose to show/hide the AI-Powered Summary for the video using the toggle.
This summary will also be visible on the playtest overview page on the video blurb under Video AI Analysis.
First Findings: Your AI-Powered playtest report
The report can be viewed on the playtest overview page in the First Findings tab.
How does First Findings AI-Powered Analysis work?
We are currently using the OpenAI API to analyze playtest video transcripts and survey responses (when PlaytestCloud's survey tool is used) to generate annotations, summaries and the playtest report.
As your playtest results come in, your analysis configuration, video transcripts and survey responses are used to generate annotations for each video and a short paragraph that sums up the experience each player had during your playtest.
As a separate process, the First Findings: AI-powered playtest report is generated through a two-step process:
Analysis: the AI analyzes the transcripts and survey responses from each player along with your configuration to identify and categorize key trends and themes.
Report Generation: Next, it synthesizes these observations into a comprehensive report that includes the following sections:
- A general overview with positive and negative highlights
- Categorized themes and insights
- Actionable advice for developers
How do I customize the First Findings report?
Once all your playtest results are in, you can customize the First Findings report by clicking on Customize Report on the First Findings tab on the playtest results page.
Regenerating analysis:
In the Customize Report panel you can modify the analysis configuration and then regenerate the analysis. You can also assign a name to the analysis before triggering the regeneration.
Your previous reports will still be available even after a new one is generated.
Selecting the Active report:
The analysis configuration you specified when ordering the playtest is used to create the System Report that is generated automatically once all results are available. When you generate a new report, it automatically gets marked as Active and is shown by default.
If you would like to change the active report, select the report from the dropdown and click on Use This Version
Regenerating AI video annotations and summaries
You also have the option to regenerate the AI video annotations and summaries; however, all existing AI annotations and summaries will be destroyed and you will not be able to access them again. We recommend using this option only if your configuration (for example game description and research questions) has changed a lot from when the playtest was ordered.
Note:
Only AI-generated annotations and summaries will be replaced with new ones when you select this option.
If you would still like to keep a copy of the existing AI annotations and summaries, they are available for download in the Annotations export (.csv & .xlsx) in the Downloads tab.
What playtest data is used to generate First Findings?
The First Findings AI-Powered Analysis is based on the transcripts of the playtest videos, the survey responses from players (when PlaytestCloud's survey tool is used) and your analysis configuration (game description, research questions and categories). We also share the game name that was added while creating the playtest with the AI to give context on the playtest.
No other information regarding the player, game studio, or game is shared with the AI, and no personal data is shared alongside the transcript and survey responses.
We currently use OpenAI API and have not opted-in for playtest data to be used for training (OpenAI's data usage policy). No personal data of players or our customers is shared with OpenAI.
As the field of AI is rapidly developing, we are constantly evaluating new AI systems to improve AI-Powered Analysis. If we switch to another solution, we will ensure that similar guarantees on data use are in place.
Will First Findings AI-Powered Analysis work for Bring-Your-Own-Players?
Yes, First Findings AI-Powered Analysis is available for playtests that use our Bring-Your-Own-Players (BYOP) feature.
If your BYOP playtest is happening in a language other than English, please contact us at hello@playtestcloud.com so we can make some adjustments to our internal configurations.
Will First Findings AI-Powered Analysis require changes to the Terms of Service or DPA?
No, using First Findings AI-Powered Analysis is possible under the existing agreements that we operate under with our customers. Your data continues to be handled securely and confidentially, in line with our agreements. Should there be an exception due to custom legal agreements in place between PlaytestCloud and your company, we will proactively reach out to you and you will have to opt-in to the use of First Findings AI-Powered Analysis.
Regarding Bring-Your-Own-Players and PlaytestCloud's status as a data processor in our relationship with our clients: The AI provider we use (currently OpenAI) does not receive any personal data from us during the processing of the videos, meaning they are not a sub-processor under our DPA.
Please see our answer to What playtest data is used to generate First Findings? as well.
What if the AI analysis doesn't make sense?
There is a certain amount of uncertainty and randomness involved when working with any AI. On the off-chance the results of the analysis do not make sense or lack context, we have two options:
Regenerate the analysis using the customization features and consider adding more details in the game description and research questions and then regenerating the analysis.
Share your feedback on why the analysis isn't helpful: We are always interested in improving the instructions we use for AI-Powered Analysis. We can look into making a general change to our instructions or making a modification for your playtest and re-running the analysis. Reach out to us at hello@playtestcloud.com.
Can I run First Findings AI-Powered Analysis on older playtests?
Yes, this is possible with a special request. Reach out to us at hello@playtestcloud.com and let us know which playtest you would like to have analyzed.
How do I enable First Findings AI-Powered Analysis for my organization?
First Findings AI-Powered Analysis is available for all current subscription plans, the Starter Plan, and one-off projects. It is not available to Booster Packages, Professional Plans (2019), and Legacy Playtesting Credits users.
If you're using one of the plans that do not have access and would like to gain access, please reach out to us at hello@playtestcloud.com.
Customizing First Findings is available to game studios on an Indie Pass, Professional Basic, Professional Advanced or Enterprise subscription. Customizing First Findings is not available on the Starter Plan.
Important: If your account should have access to First Findings AI-Powered Analysis but you do not see it in your account, please check with your organization's admin.