Sorry for the downtime folks! I'm on leave and busy with family stuff. @blinks stepped up and solved it. And since he's amazing, he wrote a post-mortem: https://gist.github.com/blinks/dabac9fb850ad00941f9a50b882a0e38
Bad news: I was tagged in to a thread with some of my least favorite people to talk D&D/DW.
Good news: Because people were wrong on the internet I migrated some 8 year old posts on D&D and AW-style Principles and now you get to read them.
Supporting dice.camp: a few people have asked about how to support dice.camp. The short answer is you don't have to. But if you want to help us meet our costs (about $20/month) we now accept contributions at http://ko-fi.com/dicecamp.
@Slybebop Naw, it's still not noticeable.
Best guess on problem: something went wrong in the database, which led to Mastodon being stuck in its regular cleanup task using the entire CPU.
The best way to get the db out of this state was the migration that happens as part of upgrades.
These are all wild guesses, nothing concrete enough to track as a bug on the Mastodon codebase.
And we're back!
Last night around 1 AM PST something maxed the CPU on our tiny virtual server. It was pegged so hard the machine was unresponsive. I restarted it, and it didn't come back up cleanly. Unable to get it to settle down, I went back to our last snapshot, which happened just before the machine went wild. (Maybe related to why it went wild?)
I was unable to stand that up either, and ended up upgrading that snapshot to the latest Mastodon code, which somehow unstuck it.
@briecs shouldn't be. What error is he seeing?
@not_another_mike notification emails? Registration emails are sending, like always, though a mail service which can in many cases get filtered. But they are still sending.
@gamethyme no problem. Had to step away for a meeting in the middle and started late, it should have been much shorter.
Low-entropy entity. Dungeon World co-designer. Senior Test Engineer at Google. He/Him.
A Mastodon instance for tabletop gamers.