“Secret Lair Survey” Survey Results

I was curious, and created a bad survey to try to answer my curiosity. Here’s the data, why the survey was bad, and my attempt at a better one.

TLDR: MTG sent out a survey about possible future directions for Secret Lair subscriptions and pricing. I was curious who received the survey, so I sent out my own survey about their survey. Here is the raw data I collected as a CSV, minus contact info if it was provided. I didn’t do a great job following up on this one. I’ve also made a better version of the survey, trying to learn from my mistakes.

About three weeks back, a lot of people who had purchased Secret Lair products received an email asking them to take a survey about Secret Lair products. I did not receive this email, but I heard from friends who did.

I don’t have any screenshots of the survey (if you do, please send them over to me, I’d like to include them in this article), but I reached out to a few of the folks who received the it. The general gist I got from talking to folks was something like this:

  1. The survey contained a variety of offers, most related to different subscriptions/subscription offers.
  2. The specific pricing on these offers ranged price based on the survey, and some folks who opened the survey multiple times stated that they saw the amount change, which implies there was some A/B testing going on regarding price point.

Regarding the specific offers themselves, there were a few different types I saw mentioned. These included same day shipping on orders, a discount on purchases, and subscriptions to all Secret Lair products for a year.

Now, all of this is very interesting, and I’d write about it more if I actually had the link to said survey, and screenshots. But I don’t. What I’m interested in is who WoTC sent the survey to. It seems like they were primarily targeting individuals who spend large sums of money on Secret Lair with this survey.

So, I did what any bored motherfucker who thought they could get a clickbait article for their blog out of it reasonable person would: I put up a Google form, posted it to Reddit and Twitter, and waited for some responses.

Unfortunately, I don’t think I got enough responses to reasonably answer my question, but I did promise to release my data in those posts, so here it is in csv format. I’ve removed the column that included contact information, but left everything else.

I do want to quickly go over why I don’t believe this data can be used to support my hypothesis which went something like “WoTC was specifically targeting folks who had spent lots of money.”

Lets start with the big one: this is an incredibly small set of responses. I’m incredibly grateful to the folks who took the time to fill it out, but 38 data points is nowhere near enough to begin drawing meaningful conclusions, and not everyone who responded had even purchased Secret Lair products.

In addition to this, this survey was from a specific subset of the population of folks who might have even seen it, i.e., Twitter and Reddit users. So we have a fairly heavy sampling bias to add to that as well.

(I think it’s actually fairly easy to manipulate this data to make the argument you want. For example, if you look at the average spend of folks who received the survey, they spent more than folks who didn’t. But if you look at actual numbers of folks who bought Secret Lair products and received the survey versus those who didn’t, there’s no clear cutoff. This is also why I’m not comfortable drawing conclusions with this data.)

On the other hand, I think there are some valuable lessons to be learned here, at least from my side. Let’s go over them briefly, shall we?

  1. Have a testable hypothesis before you randomly ask strangers to give you info. Pretty sure any science or stats teacher would smack me for what I did here, at least in terms of going “Hey, lets just gather a bunch of data, and then think about it.”
  2. If I’m going to try to do something that is time sensitive, I need to actually move quickly. Plenty of people gave me contact info. I didn’t reach out to them, partly because I had other work stuff, mostly because the new Path of Exile league came out, and I started playing that non-stop. (It turns out that doing meaningful data analysis and journalism is hard, who would have thought.)
  3. Reach out to the company involved. I probably could have just emailed WoTC. I mean, I doubt they would have responded, but who knows. It couldn’t have hurt to have tried.

So, in the spirit of being curious if WoTC is about to lean hard into whale fracking learning from my own mistakes, I’ve created a brand new survey that attempts to fix the problems with the first one. This time, we’re only interested in two things: if you received the survey, and how much you spent on Secret Lair in 2020, and 2021.

You can fill it out here.

Again, I greatly appreciate anyone who takes the time to do so, and while I doubt this second survey will hit the numbers required to do meaningful analysis, I’ve tried my best to fix some of the flaws I noticed with the first one. And just like the first one, I’ll release whatever data I get publicly.

Micro Mages

Small. Fun. And I was going to say short, but then world 4 and the world 4 boss happened. So calling the game short would be a lie.

There are a lot of interesting things that can be said about Micro Mages. For example, it came out for the NES… in 2018, which is at least a little bit after that particular console stopped being manufactured. There’s some cool stuff about the game being able to support four players via some trickery and other stuff. It also got physical cartridges manufactured, again, years after the discontinuation of said cartridges.

Of course, having zero appreciation for impressive technical achievements as I do, I don’t really care about any of those things. I’m here to answer a different question: “Is Micro Mages fun?” My answer is “Yes, now, can I go back to bed?” I’m told this is apparently not a sufficient enough answer to count as an entire article.

Okay, so what is Micro Mages? Well, it’s a fairly small vertical platformer. The game has four areas, with three levels each, and a boss at the end. (And two bosses, sorta, at the end of fourth area.) You control one of the titular miniature magi in their quest to get through all the levels, via wall jumping, shooting projectiles, and trying not to die. You die in one hit (if you haven’t picked up any powerups), making avoiding death a bit more difficult than you would think. Once you beat the game the first time around, there is also a hard mode, in which you replay the same worlds, but the enemies get additional behaviors/attacks, and the rate at which the game auto-scrolls up gets faster.

Why yes, I did just take all of my images from their press kit. Except the last one of course.

This is all just so much fluff in describing the game, and the reality is that while Micro Mages is really simple, it’s also quite fun. Everything about how it controls and plays feels well thought-out. A few things of note for me were how fluid and accurate the wall jumping felt, along with the fact that projectiles could be fired in all of the compass directions, and almost always went where I wanted them to go. In addition, the range on said projectiles was generous, avoiding the classic “The projectile despawned right before hitting the enemy and now you’re dead” moment.

The powerups are fairly plain, but they do what was intended. The game only really had one instance of mechanics screw/death is the best teacher. (Looking at you, giant floating skull that speeds up if you hit it with a projectile and can’t be killed.)

I remember there being a really cool article about how the game saved space by using mirrored images to construct the mages and the bosses, but I can’t find. If I track it down, I’ll link it here.

Outside of the final world and final world’s final boss, I would say the game isn’t too difficult, and it’s also short enough to be worth playing. And, like many other things I write about, if you purchased the itch.io racial justice bundle, you already own it!

Did I beat it from world 1 to 4? No. But did I clear the whole game? Technically still also no; I didn’t beat hard mode. But I did beat it in normal, and that’s what really matters.

If you didn’t, it’s $10 on Steam, the same on itch.io, and 45 European Monetary Units for the physical cartridge, complete with game manual, and all that other good stuff. I have no idea if they ship worldwide or anything though.

Ed Note: I was planning on having this article be very short, as part of a meta joke about how small Micro Mages is. Except then the final level absolutely kicked my ass for quite a bit, and I had to abandon that plan.

Ed Note 2: You have no idea how incredibly pleased I am with the phrase “Miniature Magi,” and even though it’s not as clever as I think it is, it fucking kills me that no one will see it.

Chess Evolved Online

Did you ever look at Chess and think “Wow, I really wish Chess was more complicated, and that you could power up your units?” Well boy do I have the game for you.

I think Chess Evolved Online (or CEO as I’ll abbreviate it) is neat. If you like Chess, it’s worth checking out. If you don’t like Chess, you can save yourself the time of reading this article by closing your web browser and doing something else. Okay, lede dealt with. Time for a diatribe.

I write this blog about games at least in part because I consider myself pretty good at them, though in retrospect, “Experienced” might be a better word. I play a lot of games. There are many things that I’m not very experienced with, and so I don’t really comment on them, such as international relations, and agricultural policy. I still have opinions on them, but I don’t think they’re valuable to share.

I mention all of this because CEO has reminded me of something I forgot: I really, really suck at Chess. Or at least I really suck at it relative to people who play Chess at any level of actual competition or play.

I bring this up because after playing 12 hours of CEO, I couldn’t tell you if the game is balanced, or fair, or a good Chess equivalent or what. I could probably play 100 more hours, and I still wouldn’t have a good idea.

So instead of talking too much about that aspect of the game, I’m just gonna quickly list a few of the mechanical innovations CEO makes vs an ordinary Chess game, talk about the game’s meta progression structure and my thoughts on that, and then wrap up with a link to the game. I’m not gonna talk about balance or fairness, or whatnot, because I don’t think I’ll get any of it right. I’m also going to assume anyone reading this knows how Chess is played, and if you don’t, allow me to link you to the Wikipedia page on the game.

I did not win this.

So, new mechanics. Unlike in Chess, CEO has a different set of win/loss conditions: Morale. Each of your pieces has a value associated with it, which as far as I can tell, generally scales to its power/utility. The queen is worth 21 points, a bishop is worth 12, a rook is worth like 13. You get the idea. When your piece gets taken, you lose that much morale. If your morale hits zero, you lose.

Your king, on the other hand, is special. When your king gets taken, you lose an immediate 25 morale, and then an additional 3 morale per turn. The fairly obvious result of this is that while losing your king will eventually lead to a loss, unlike in standard Chess, it’s not an immediate loss. This has some interesting implications, like being able to trade kings, and also ending up in situations where you have to decide if sacrificing your king might be worth it.

The other special factor controlling the game is “move decay.” After turn 50, each player loses 1 morale at the start of their turn. This effectively puts a cap on how long games can go, and also means that in a game of attrition, whichever player can take a take an early advantage is likely to win. As far as I can tell, there are no draws in CEO.

There’s one last big mechanic in CEO: time. I’m not familiar with professional/semi-pro Chess, but a few quick google searches make it look like the game is limited is to about 90 minutes for your first 40 moves or so.

As you might guess, CEO doesn’t really go in for that. Instead CEO has two formats: CEO Blitz and CEO Standard. Standard gives you 30 seconds a move, after which you have a pool of 4:00 minutes. Blitz gives the same 30 seconds but with a pool of 2:00.

The result of all of these changes, at least to me, is the game feels far faster paced and bloodier than standard Chess. At the same time, the fact that losing your king isn’t a loss means that games can turn into brawls far faster than standard Chess.

So, those are the general changes to the game’s structure. Now let’s talk about the army building. Yes, CEO has army building.

There are a few factors that go into army building. The game has two types of pieces: champions and minions. In general terms, you can only place minions in the front row, and only place champions in the back row. Minions are weaker than champions, and usually (but not always) have a promote ability. This is where another interesting change comes into play: pawns always promote to bishops. This opens up some interesting space for weaker minions with stronger promotions.

Pieces also have a supply cost, and your army has to be under your supply limit. You increase said limit by reaching a higher rank. I don’t love this too much, but I’ll talk about why later.

Everything else about the game feels like a fairly unique attempt to fix some of the problems that Chess has. But how you obtain and upgrade pieces for your army feels fairly standard. Because, of course, pieces can be upgraded.

The game has two currencies, gold and gems. You earn gold by playing games, and you earn more by winning. You get gems by either exchanging gold for gems, completing various objectives, or opening random boxes.

Or spending real money.

And this is why I say it feels standard: The game loop becomes a pattern of grinding for currency to either buy units straight up at a in-game shop, or buy random booster boxes of units in the hopes of getting something you want. At the same time, you use duplicate units to upgrade your existing units.

Tell me this doesn’t look like a mobile game UI.

And this is where the game started to annoy me. Perhaps it’s because I’m not great at Chess, but all of a sudden the game starting feeling like a mobile game. I was no longer logging on to build unique armies or try interesting strategies. I was logging in to open daily boxes, playing ranked for currency to try to grind more boxes, and just generally playing the game solely to get more units. Instead of trying to build unique armies and innovate, I played the standard Chess army because I was more familiar with it, and I cared more about winning than experimenting.

Now, if I was better at Chess, this might not be true. Perhaps if I was a stronger player, I would find it easier to win, and as such be more interested in some of the other systems that make up the game. As it was, though, I ended up feeling like I wasn’t playing for fun, or to use the game’s unique systems, but to just grind to get more pieces. And this is where I stopped playing.

In summary, CEO’s actual gameplay containes a bunch of massive changes to the core structure of Chess, to try to make it more exciting and interesting, while also speeding up the pace of the game. But the meta-progression structure that exists around the gameplay feels like a standard mobile game, and it feels bad. If you end up matched against AI or players with higher ranking than yourself, in addition to likely being better at the game than you, they also have a larger supply pool to pay for their army. Even if you’re both playing identical armies, if they’ve upgraded their pieces, they have access to options and moves you don’t.

And that doesn’t feel great. And while I suspect that the actual mechanics of things like supply/value on units are probably balanced, it still feels bad to get decimated by someone with a unit you just don’t have.

Ed Note: The poster child for this particular experience is the ninja, a unit that feels like a knight on steroids, with the ability to take pieces in all directions that are adjacent, while also having a unblockable jump.

This just feels like bullshit.

Chess Evolved Online is free on Steam, with in-app purchases to buy additional rubies, which in turn get spent on buying units/random booster boxes of units. The game makes a bunch of really interesting mechanical changes to the base game of Chess, but it makes them parallel to a meta-progression structure that, for me, made the game feel like a grind. I still think it’s interesting enough that folks should check it out, but I feel like without a strong interest or background in Chess, you might end up having a similar experience to me.

Dota: Dragon’s Blood – Spoiler-ish

What the ever loving fuck.

I have played a lot of Dota. Like a lot. Not as much as some people, but still over several thousand hours. I also like anime, animated shows, and have low expectations for non-game media based off of games.

As such, I think I might be the exact audience for Netflix’s new show Dota: Dragons Blood. As I sit here having finished it, my only thought is:

What the fuck did I just watch?

No, seriously, if you know, can you please tell me?

I really like Dota 2. I know my Dota 2 lore. I may be one of 12 people who enjoyed the Dota CCG, Artifact, and I’ve read all the comics. And I’m still bewildered.

The best way I can put it is that I had a set of expectations going into this show, and it did not match them. I expected a fairly generic fantasy world, maybe passable characters, and a generic but acceptable plot.

What I got was a fantasy world that feels like it’s based off a world bible that no one but the producers has ever read, surprisingly solid characters, and a plot that waltzes its way, high as a fucking kite, hither and yon, and yon and hither, giving absolutely zero fucks about things like “pacing” and “tonal consistency.” One moment, we’re in a mystical library. Next moment, there are a bunch of elves having a foursome. Then murder. Now war crimes. I think this was all the same episode.

I’m honestly not sure what to think.

I do have one absolutely massive complaint about the show: it wraps up zero of the plot points it introduces, even regarding its main characters. And honestly, that’s kind of a shame, because I would be surprised if the show gets a second season to wrap them all up. Valve historically sucks at continuing projects that don’t go gangbusters, and also the show is weird as fuck.

Edit: Since I wrote this, they’ve confirmed a “Book 2” is being made. I’m still not sure why.

The show has a lot of things I can see potentially turning folks off it. Gratuitous nudity (mostly male). A lot of violence. Swearing that some folks will see as “Well, that’s just how folks talk,” and others will see as “We’re so grimdark and cool.”

I’m not sure how you’re supposed to evaluate television. Would I tell other people to watch the show? I mean, maybe. Maybe if they already like somewhat edgy animation. Did it succeed in making me care about the characters? I mean, yes, otherwise I wouldn’t be upset with how it ended, wrapping up zero plot points, and setting up for a second season that might not ever exist.

While the motivations of the characters themselves and their actions make sense in context, the world they’re taking those actions in often feels a little pants on head crazy. While the animation is really nice at times, the end result feels like reading an independent comic book with a creator who has equal amounts of technical artistic skill and ketamine at their disposal, and has primarily opted to use the ketamine while doing the writing.

If this for some reason has made you want to watch it, it’s on Netflix.

The show can also be found in my haunted dreams, but I don’t think there’s a subscription service for that yet.

Racial Justice Bundle – Didn’t make the Cut

Wow, it’s been a while since we’ve done one of these, huh? I finally dug back into the ol’ backlog of games this weekend, looking for more treasure in the large pile of…. stuff.

Stuff that people probably worked very hard on, but still wasn’t actually all that fun or interesting.

Or (at least in my subjective opinion) wasn’t “Good.”

As always, I encourage readers to download and play these games yourself. Don’t just take my word for it if I call something stupid garbage! Experience it yourself. Join me in the digital equivalent of dumpster diving, because as they say, one man’s trash can be another man’s treasure.

Babysitter Bloodbath

I’m not sure what to say about this game, other than: it has surprisingly nice polish and production values, but I still didn’t care enough to want to keep playing it. The whole thing gave me a super old-school Resident Evil vibe, and frankly, I don’t much like horror. So after the second time the controls bugged, and I couldn’t turn left or right, I was more then happy to put it down, and pick up something else.

Diaries of a Spaceport Janitor

Diaries of a Spaceport Janitor is a game about being a spaceport janitor. You burn trash, try not to vomit everywhere from eating garbage out of a vending machine, avoid getting shaken down by cops who eat your money, and also search for a way to remove a giant weird skull that perpetually floats behind you screaming.

Okay, so my description sounds kind of cool, but the game just feels boring. You wander around, trapped in a loop of never having enough money to buy anything, within a maze-like city zone, all the while just generally having everything be shitty and sucky. Oh, and while torching garbage.

Maybe the intention of the designers was to create an atmosphere of boredom and fatigue, caused by doing the same thing every day, and feeling like a hamster trapped on a wheel. I don’t know. Apparently the giant floating skull is a metaphor for depression? Not sure I really connect with that one either. Point is, if the intention was to make something that feels boring and dreadful, they succeeded. I am bored, and I would dread ever playing this thing again.

Some neat art and a little bit of neat worldbuilding though.

Tonight We Riot

Unfair enforcement of laws in regards to minorities, and those with sexual or political orientation differing from the mainstream are a big issue. Lack of police oversight and accountability are serious problems. Capitalism has significant flaws that are having lasting impacts on our society.

I don’t think the proposed response of Tonight We Riot, which appears to be murdering riot police with cinder blocks, and torching bankers with molotov cocktails is a great solution to either of those problems.

So yeah. I don’t love the theming. And I also don’t love the controls. Many of them feel fairly wonky, and the targeting on things like ranged weapons, and controls for moving your fellow workers also don’t feel great. I played through the first big boss and called it a day.

Dorfromantik Demo

Dorfromantik ends up in a “Didn’t make the cut” article not because it’s bad, but because it’s a very barebones demo that while nice, didn’t compel me to rush to buy the full game. If there was any single item on this list right now that I would actively encourage other people to check out, Dorfromantik would be it. Given that the full game sits at 2400 reviews with an average of overwhelmingly positive on Steam, I think people other than me might like it.

Just a hunch, y’know?