Breaking down player choices in Scarlet Hollow Episode 2
For those of you who aren’t in-the-know, Scarlet Hollow is an episodic narrative horror game currently in early access on Steam and itch. A couple months after the release of each episode, we send out surveys to get a sense of what choices our players have been making, and then digest that info into a blog post that shares both the data we’ve collected and our insights on how that breakdown influences our design philosophy.
You can read the first of these posts, which analyzes our trait-based roleplaying system, here. If you haven’t had the chance to play Scarlet Hollow (Episode 1 is free!) or read about our roleplaying system, now’s a perfect time!
We also share these a week early with our $5 Patrons, so if you’ve got the change and would like to help our small indie studio stay small and indie, your support is greatly appreciated. Now onto the good stuff…
This time around, our survey got a whopping 457 responses, a huge step up over the 72 responses we got on the post-Episode 1 survey we sent out after our Talk to Animals patch. It’s worth noting that for most of the questions in the survey, we’ve asked players to answer based on what they consider to be their canonical run, since Scarlet Hollow has considerable replayability. In fact, that replay value might be the best place for us to start →
Over sixty-percent of our respondents have played through the first two Epsiodes of Scarlet Hollow at least twice. Over 22% of our respondents have played through the game at least four times.
While it’s likely that a survey like this has a self-selection bias towards players who have completed more than one run (higher engagement, etc), it’s worth noting that the responses here aren’t that far off from the playtime data we’ve gotten from Steam.
Both the mean and median playtime for the Early Access build is over twice that of our Episode 1 page, despite Episode 2’s runtime being roughly equivalent to Episode 1’s (60–90 minutes, depending on reading speed and how completionist a player is about Explore options and seeing as much content as possible.)
This effect is even more pronounced when you account for save game imports — save-states in the Episode 1 build are automatically accessible when you load up the early access build, which means a large number of our players started their playthroughs with Episode 1 already finished.
Traits
The canonical trait breakdown between episodes was almost the same as the last time we surveyed our players, with one exception — Mystical saw a significant jump in popularity. In our Episode 1 survey, traits fell into essentially two tiers. Talk to Animals and Keen Eye were petty much tied for most popular, with each one appearing in almost half of all canonical playthroughs. And everything else was tied for third place.
This time around, we saw three different tiers in terms of pick-rate: Keen Eye and Talk to Animals were within a standard deviation of each other, and Mystical is in the second tier by itself, roughly one standard deviation off from the first bucket. And then the remaining traits (Book Smart, Street Smart, Hot, and Powerful Build,) are in a third tier, with the first three closely clustered together, and Powerful Build around half a standard deviation less popular than those three.
Powerful Build being the least popular trait is a little surprising considering it provides a pivotal third option/easy way out to the Big decision in Episode 1, which is something that, at this point, we assumed most players were aware of. Powerful Build’s unpopularity became even more pronounced when I pruned our survey results and eliminated responses that selected more than two traits. Most traits lost a little over 10 entries in that scenario, but Powerful Build lost fourteen, which, as the least popular trait impacted its total pick rate even further, leaving it at an 13.2% pick-rate and almost a standard deviation away from the next least-popular trait. There aren’t a lot of be-muscled beefcakes running around Scarlet Hollow, it would seem! ¯\_(ツ)_/¯
Overall, the result of pruning was that everything took a hit in pick percentage (obviously), but some traits, like Powerful Build, took bigger hits than others relative to their standings. If you’re breaking things out into tiers, we potentially have four here. Mystical is over a standard deviation under-picked than Keen Eye and Talk to Animals, and Powerful Build is just shy of a standard deviation away from the 20% trio.
- S: Keen Eye, Talk to Animals
- A: Mystical
- B: Street Smart, Book Smart, Keen Eye
- C: Powerful Build
We also took a quick look at the frequency of trait pairings. Mystical;Talk to Animals and Keen Eye;Talk to Animals were the most popular, followed by Mystical;Keen Eye and Keen Eye;Book Smart. Unsurprisingly, many of the least popular builds involve Powerful Build.
For the most part, the most popular traits are part of the most popular pairings, and the least popular traits are part of the least popular pairings. Of interest, Mystical;Talk to Animals is the most popular combo despite Keen Eye being the most popular trait, likely due to a popular interaction between the two traits (Ditchling whispers are in plain English.) Powerful Build;Hot also outperforms the relative popularity of each of those traits individually ;)
Also of note is that all 21 trait combinations saw some amount of play-time in respondents’ canonical runs, though some, like Powerful Build;Book Smart, Powerful Build;Mystical, Powerful Build;Street Smart; and Street Smart;Book Smart saw very little play.
Choices
Trait breakdowns are all well and good, but I know that the real reason you’re here is to see how your decisions stacked up against others. A brief note: I neglected to collect data here on the big Episode 1 decision. Going off of trait data, though, you can assume that both Gretchen and Duke survived in 13.2% of playthroughs (Powerful Build pick-rate), and going off of an ancient Twitter poll, 20% of non-Powerful Build players saved Duke.
At the end of Episode 1, players get the choice to spend the night with their new friend (maybe flame?) Stella, or return home to their cousin’s Estate. From watching people’s playthroughs on Stream, we thought that this would wind up pretty lopsided in favor of Stella, but it turns out there was a pretty close to even split that actually wound up favoring Tabitha.
This is despite several things working against the player returning to the Estate, including:
- Wanting to pursue a romance with Stella
- A double confirmation screen in the form of Stella asking you if you’re sure you want to go back.
- Going back meaning that you have to trek through the woods alone at night.
- Tabitha is an asshole.
A huge amount of work went into making two separate introductions to Episode 2 based on where you slept, and it’s great to know that a large portion of the player-base chose each option.
There’s no immediate downside to getting coffee with Avery, but it is optional content, and a focus of Scarlet Hollow’s game design is choosing what not to share with people. Part of Avery’s character revolves around being an outsider — people are friendly with them, but they haven’t managed to find a niche in town, and part of how we’ve chosen to represent this arc is by making the player make conscious decisions about whether or not they’d like to spend time with Avery or invite them to events.
This response lines up well with our achievement data on Steam, where 76% of players got coffee with Avery (that metric includes players who haven’t finished their playthroughs yet, so you would assume it to be lower than the responses here.)
Next up, we asked players which of the three books they read in the library.
It turns out that most of you are nerds. Only a tiny handful of players read less than all of the books available. We’ll take that to mean the library scene is working as intended — a lot of thought went into how we could best make researching paranormal events in a library engaging and immersive within a visual novel.
To give some extra context to the remaining questions, we’re going to go a little out of order and look at player responses to our romance questions.
Between Episodes 1 and 2, Stella significantly solidified her lead as the most popular romance option, with second place being held by Nobody (fair) and effectively a four way tie between the other options. That being said, given the discrepancy of responses between the Episode 1 and 2 surveys, a direct comparison of those data sets probably isn’t the most helpful. Thankfully, we thought about that before sending this survey out.
Most of the respondents who said that they changed their minds selected Avery, Oscar, or Kaneeka as their preferred romance option. This is unsurprising given the additional content those characters received in Episode 2, but it is surprising given how much Stella furthered her lead between the two surveys, and given how much the other romance options lost ground.
I have a few thoughts on why that might be the case:
- Changing Stella’s sprites made people more likely to romance her — new players would have only known Stella’s new sprites, and returning players likely played with the new sprites at some point between the first two episodes.
- The demographics of our player-base might have changed between episode releases.
- The first survey was too small a sample size to make a meaningful comparison. (The boring stats answer)
These numbers will probably continue to change as we release more content, especially given that players haven’t met Reese, and have had limited interactions with Oscar and Avery.
Now that we’ve got some context for romantic inclinations:
About 2/3 of our players went with Stella to the mines, while a little over 1/3 went with Kaneeka. However, if you account for Stella’s 40% pick-rate as a romance option and Kaneeka’s 10% pick-rate, and assume that players choose to travel with their would-be flame, both of those options are even.
Filtering this question based on romance choices mostly supports the above hypothesis, though the numbers aren’t quite even, and there’s uneven bleedover with more Kaneeka-mancers traveling with Stella than the other way around. This is probably measuring one of two things:
- Stella traveling alone in the woods seems dangerous, so a player interested in Kaneeka might still feel like they have to travel with Stella to protect her.
- Players who changed their mind from wanting to romance Stella to wanting to romance Kaneeka made that choice later on in the episode.
Checking that second point changed the numbers a little, but at this point we’re dealing with small enough samples that the data doesn’t really mean anything.
Players were torn 60/40 on what they wanted to do re: Tabitha in the mining camp, and 1/3 of the players who decided to avoid Tabitha decided to leave before getting caught. We were happy to see this split since, like the library, it validated a lot of game design decisions for this scene.
Players were a lot less split about whether or not they wanted to call Tabitha before going into the mines, with over 80% of our respondents calling her. Another 77.5% went into the mines with Stella and Kaneeka, either because they didn’t call Tabitha or didn’t want to wait for her.
The Big Decision in Episode 2 is partially overshadowed by the fact that Keen Eye, the trait that lets you get out of it, is our most popular trait with a 44.7% pick rate. There’s a bit of a discrepancy between that pick-rate and the number of players who selected its correlated option here, and filtering out Keen Eye respondents, there were still 33 people who picked all three.
Removing the third way out entirely, we’re at a 90% pick-rate for Rosalina being injured, and a 10% pick-rate for leaving Alexis and Becka behind. We decided to be a little tame with the visuals re: Rosalina’s injuries in Episode 2. It’s possible that this tameness underscores the severity of what happened to her, and that if that severity were more clearly communicated, there might have been a more even balance for this decision. Before we release Episode 3, we might go back and patch that scene so its art is more reflective of what has happened to her. Regardless, the side effects of that choice will be more pronounced in the next episode.
An interesting slice of the above chart is how players’ decisions about calling Tabitha correlated with their decision in the mine. Regardless of whether or not they called Tabitha, going into the mines with Stella and Kaneeka meant that players were even more likely to go after Becka and Alexis, while waiting for Tabitha made players 50% more likely to leave the mines with just Rosalina.
And that’s it for now! Thank you so much for reading and we can’t wait to show you what we’ve been cooking up with Episode 3!
If you haven’t played Scarlet Hollow yet, Episode 1 is free on Steam, and you can purchase or wishlist the full game (in early access) here. You can also follow us on Twitter, or join our Subreddit or Discord.
If you like these posts, please consider following this blog on Medium or supporting our studio on Patreon, where you can get early access to more posts like this, and even weigh in on what you’d like us to cover next.
Until next time!
