Sueness and Popularity: Mistakes

I’m (finally!) scoring the last of the webcomics for my Sueness and Popularity project.

But I’ve just realised a major flaw in my experiment plan: I haven’t specified which characters I will score as “Fan Characters & Newcomers”. The test has a separate section for these characters, and I know from my first scoring project that the choice of section makes a big difference to a character’s score.

This is fine for fan characters, since it’s usually fairly obvious when a webcomic is based on an existing story. But for “newcomers”, it’s a difficult judgement call. As a reader, I can’t reliably tell if a character is a new one the author just made up, or if they were planning to introduce them all along.

I could pick a rule: for instance, “any character who doesn’t show up in the first storyline is a newcomer”. But for something that could have such a big impact on my results, I don’t want to rely on such an arbitrary choice.

Alternatively, I could ignore the original-character/newcomer distinction, and just score every character on the “Original Fiction” scale. This avoids arbitrary cut-offs, but it also means I’m not applying the test properly.

Neither choice really makes sense, so I’m going to re-plan my experiment to try both:

Continue reading

Sueness and Popularity: Sample

(Continuing the project from this data collection post.)

I’ve now decided on my 50-webcomic sample (download the full list).

It’s one month today since I sampled the first one, so it’s time to start reading and scoring them.
Before I do, though, I’d like to say what I’ve noticed so far:


There are several comics in that list I might not be able to score. (I should have made a plan to handle this, but it’s too late to change the experiment rules now.)

No Regular Characters

Two comics (The Science of Cookies and 3 Frame Movies) don’t seem to have any recurring characters (i.e. no-one shows up on more than one page). By the rules I’ve set out, this means I can’t score anyone in the comic, and therefore can’t analyse it.

This is probably fine — if a comic has no prominent characters, I can’t really measure it for Mary Sues in a useful way.

Too Many Authors

Another two comics (The Adventures of Fez Boy and Dino Duck) are “interactive” comics like MS Paint Adventures (readers suggest actions for characters, and the author picks one and describes the result).

It’s hard to argue these authors bear all the blame for any Mary Sues — or even that it’s a problem at all. Author self-insertion is one thing, but audience self-insertion (or is that just “identifying with the character”?) is quite another.

Fortunately, these comics shouldn’t have too many people making suggestions, so I can just spread the blame equally: if someone’s suggestion gets used, count them as an author.

(Needless to say, I’ve deliberately not submitted any suggestions to these comics myself).


The next time I do this, I’ll need to have a list of rules to exclude comics (e.g. must have a regular cast, must be fictional, must be in a language I read), and decide how (or if) I want to find extra samples to make up for the excluded ones.

Other Complications

There were a few grey areas around what counted as a page.

Generally, I counted all images posted on the site, provided they were either (a) posted as part of the main comic, or (b) had some sort of narrative. (Title pages and side comics count, but random art pieces don’t.) In particular, I didn’t count (and won’t read) anything I had to buy to see.

Page sizes vary, both between comics (three-panel strip versus full pages) and within them (pages posted as double images). For sanity’s sake, I counted each individual blog or image post as a “page”, regardless of size.

This means page counts aren’t comparable, and I’ll only be able to compare relative counts for characters (e.g. “10 out of 100 pages” and “2 out of 20” are the same).

More updates once I’ve got some scoring done!

Sueness and Popularity: Data Collection Protocol

(Continued from yesterday’s Sueness and Popularity post)

This post is a list of all the statistics I’m going to collect on each webcomic. It might not be very entertaining; the point is to stop me fiddling with things half-way through the process.

Comic Details

  • Title
  • Website (somewhere from which the rest of the comic is reachable — preferably the home page)

Author details

For each artist, writer, etc. if several. This includes colourists.

  • Full Name, if public (The test I’m using cares about author name, but I don’t want to go stalking people just to get accurate results.)
  • Any other public names, psuedonyms, etc
  • Date they started contributing to the story (if they weren’t there from the start)

The Webcomic List Statistics

I’ll collect each of these numbers twice: once when the comic is first added, and again a month later when I score it.

If I miss a day, I’ll collect the data for that day as soon as I have time (hopefully not more than a couple of days late).

  • Nominal data-collection date (when I should have collected these stats — date added, or that date plus one month).
  • Actual data-collection date (should usually be the same)
  • Number of comic pages. This includes side stories, title pages, etc., but not non-story content (cast lists, author commentary), or guest comics that don’t contribute to canon. Text-only pages (like Erfworld often has) do count.
  • TWL stats (“views this month”, “average views a month”, “favourite of n members”)
  • Number of TWL comments
  • Number of “accusations” in those comments (posts saying the comic contains a Mary Sue)

Character Details

I aim to score every distinct character who appears in more than one comic page (if there’s only one page, I’ll score every character in it). This includes animals, AI’s, robots, etc., but not random inanimate objects (e.g. the One Ring isn’t a character).

Interchangeable randoms (e.g. mooks, or most of the stick figures in XKCD) don’t count, but named characters who never appear on panel do (if they’re mentioned on at least two pages).

As well as the score, I’ll collect this inforation:

  • Name
  • Other names (psuedonyms, superhero names, maiden names, etc.)
  • Gender (“male”, “female”, or “other”, based on how the character’s presented in the comic (so robots, aliens, etc. can be male or female if they’re presented as clearly one gender). If the correct gender is complex or unclear, I’ll use “other”.
  • Number of pages with this character (same rules as above).

Mary Sue Test score

(For each non-one-off character, as above.)

I’ll score characters based off supplementary information (commentary, previous comics with that character, etc.), even if I wouldn’t include it in the page count. However, I won’t consider (and will try to avoid reading) anything that was published after the one-month cutoff date.

If a comic is fanfic, I’ll try to research the canon enough to score it correctly — but I won’t necessarily read the whole source canon.

This might mean reading spoilers. If so, I reserve the right to either (a) put off scoring the fanfic until I’ve read the original, or (b) get someone else to score it according to this protocol. However, I don’t plan to do this unless the original is something I really want to read.

A lot of questions ask about the author’s intent or feelings. If I have that information (e.g. from author commentary), I’ll use it. If not, I’ll default to “no” when the question is just about the author’s mind (e.g. “do you see your character as a role model”), or “yes” if there’s a concrete part I can answer “yes” to.

For instance, “does your character have … an exotic name … chosen primarily because” will get a “yes” if the character has an exotic name but I don’t know why it was chosen.

If I run into anything else ambiguous, I’ll check SyeraMiktayee’s (but I won’t actually ask about it, as I don’t want to spam xir with questions). If that doesn’t clear it up, I’ll just pick something and document what I did.

Can a Mary Sue Test Predict Popularity? (Experiment Plan)

A few years ago, I calculated some statistics on SyeraMiktayee’s Universal Mary Sue Litmus Test.

This time, I’m back with a specific question: can it predict popularity?

If Mary Sues are bad writing, and the most popular stories tend to be well written, then stories with Mary Sues should be less popular.

To test this, I’m going to score 50 webcomics, and see if “Sueness” and popularity line up.

Data Collection

There are plenty of sites that track popular webcomics. However, they tend to focus on rankings (e.g. “5th most popular comic). Unfortunately, scoring the 50 best comics won’t tell me much about bad writing.

Instead, I’m going to score 50 new-ish comics.

I’ve found a site, the webcomic list, which tracks views per month (a good approximation of popularity), and also has a list of recently added comics. With any luck, this will include some terrible ones.

To prove I’m not picking and choosing what I score, I’m going to score the first 50 comics added to that page after Tuesday (12 July). I’ll give each one a month (so there’s a decent amount of content) then start scoring on 12 August.

Data Analysis

I’ve got lots of theories I want to try, and consequently lots to record.

For the popularity question, though, only two things matter: the “sueness” (Mary Sue Test score) of the highest-scoring worst character (since one Sue can ruin a whole story), and the average views per month (as of the 1-month anniversary when I score the comic).

I’ll compare these with a simple linear regression. If I’m right, this will show a negative correlation (higher “Sueness” equals lower popularity).

If I get a “p-value” of less than 0.05 (i.e. a 1 in 20 or less chance that the result was just random), I’ll consider it proof (or at least, strong evidence) that the test can predict popularity.

What next?

Before I start scoring anything, I need to define the rules I’ll use to do it.

Expect a post on Monday or Tuesday with a list of exactly what I’m going to record, and how I’m going to decide tricky questions.

A Mary Sue Test and Statistics

This is about the Universal Mary-Sue Litmus Test.

A lot has already been said about it; some disagree with the whole idea of a test, others support the idea, but debate the choice of specific questions. I’m trying to measure whether it works using statistics.

A histogram of character scores

The number of characters who scored in each range on the test. Each bar is the number of characters I scored who got a particular explanation at the end.

From the characters I’ve scored so far (about 270; see the full list), it looks like it does.

According to the current version of the test, a good character (i.e. not a Mary Sue) should score below 30. On that basis, 195 of the characters I scored are probably OK — as the graph shows, a lot of them scored between 0 and 16, and over half scored less than 21 points.

Of course, this also means 80 of the characters I’ve scored probably are Sues.

So does that mean the test works? Well, that depends…

Continue reading