Perhaps you've read our Annotations 101 article and are ready to get started enhancing your playtest analysis and note-taking with hashtags: this article will show you some tips, tricks and best practices for getting the most out of them.
When you type a hash symbol into your annotation box, you'll see a short list of standard hashtags that you may want to use. These are based on the most common uses that we have seen in the past, and work well combined with other, more specific hashtags.
Planning your use of hashtags
When you order a playtest, you will likely have a list of research questions, or just things that you want to know from running the study. Here are some tips for organising your use of hashtags around these research goals or questions.
Before you even start looking at your data, through your list of research goals and questions, and think about what you will look for in the player videos to answer these big issues.
For each research goal or question, write down one or more appropriate hashtags. For example, one of your research questions is "can players craft weapons?" For this you could use the hashtag #crafting.
In annotating your videos, you can then add this hashtag every time each you see player entering the weapon crafting screen for the first time, and then every time they successfully craft a weapon, or when they fail to craft one.
Planning your use of hashtags in this way ensures that you are addressing your most important aims when analysing the results; and reduces the likelihood that you will have to spend more time going through the videos repeatedly.
Using hashtags opportunistically
You will also most likely want to come up with new hashtags 'on the fly' for issues that you had not necessarily planned for when setting up the study. The following are some examples and tips for working with hashtags in this way.
Here is an example of where creating hashtags on the fly is useful: you notice players consistently failing to make a jump on level 3, so you add a new hashtag #level3jump to ensure that all these instances are marked consistently.
You also notice that this seems to be happening due to problems with the in-game camera, so you add a second hashtag of #camera to this annotation. This is a hashtag that you have been consistently using to mark every instance of players having instances with the in game camera.
Sometimes you may not realise that an issue warrants a hashtag until you have seen it happen a few times: for example, the first time that you see players hesitating to tap a glowing icon, you may not notice it. It's only after the third player takes a much longer time that you realise that this is a problem, and that the previous players had also demonstrated it to a lesser degree!
In the above case, you can always jump back and tag the instances of the issue that you remember seeing. But there may be value in allocating time to a second comprehensive 'sweep' through your videos to ensure that you have marked them all.
Combining hashtags
The previous use cases now provide you with powerful evidence for the impact of an issue in your game. For example, let's say you saw 20 mentions of problems with the in-game camera; your hashtag count for #camera is 20. But of these, from looking at the #level3jump hashtag, you see that 18 of those 20 were related to that single jump on Level 3. Without this information you may have overhauled your entire camera system; but in fact, it may be an issue specific to one part of Level 3.
Accordingly, don't be afraid to attach multiple hashtags to an annotation: there's no limit to how many you can use per annotation, and you can always remove them later on (it can be useful to have a round of cleaning your hashtags, especially if others will be using, reading or adding to them).
Another common use case is adding both specific and general hashtags. For example:
Players are seen expressing a desire to craft a new item, but struggling to navigate to the section of the menus where they can do this. You add a hashtag here to represent this issue: e.g. #craftfail.
However, you also want to tag this as a UI issue so that your designers can view all the relevant issues to them under a single tag. You also add the hashtag #UI.
You also created the hashtag #crafting earlier on. You add this in addition to the previous two for the sake of completeness.
Now you have three separate avenues of investigation that this particular moment could fall under when analysing your playtest: the specific hurdle of finding the crafting menu when trying to craft, the general performance of the game's UI, and the general performance of the crafting system.
Hashtags and structure
You can also use hashtags for structure. For example, adding a hashtag when players:
complete the tutorial
obtain a skill point
or upgrade their character for the first time.
This is especially useful when you want to come back and retrospectively revisit a playtest; perhaps you are making a small change to the in-game store and want to see how players navigated it in the most recent playtest. If you have all the instances of visits to the store tagged under #store, a quick click in the hashtags menu will provide direct access to that data.
Hashtags and analysis
One of the main benefits of using hashtags comprehensively throughout your annotations is faster analysis.
From the hashtags page you can extract counts of each hashtag to judge the relative frequency of occurrence of each.
This also allows you to judge how common each was among your players: for example, it could quickly reveal that a high priority issue was observed among a majority of players, and provide you with the information you need to gauge a count or percentage of players who displayed it.
You can quickly jump to the video segments for each; especially useful if you need to investigate the issue further to write it up.
You can extract video clips for each and create a dedicated highlight reel. Either use these to refer back to when adding detail and colour to your description of the issue, or share them, either in the report or directly to stakeholders.