Thoughts on judging

Whenever we go to a modelling competition, one aspect that always gets a lot of attention is the judging. It seems as though everyone has an opinion on the judging, and rarely is it the case that those with the most vocal opinions think that the judging was fine.

Now, I’m not going to get into the controversy over whether Jimbo’s P-69 Thundercat should have beaten Cletus’ Blackburn Bastard for best in show at the East Westington IPMS show three years ago, but I do think it is worthwhile to explain the difference between different judging systems and the pros and cons of each, if only to help out new people. IPMS USA is currently discussing switching from 1-2-3 to open system at their nationals, and the style of judging (if you even plan to make it a competitive thing) is one important decision for anyone looking at starting up a local show.

only-judy-can-judge-me-37511560.jpg

This guy gets it

There are a couple important caveats to this article that I would like to address up front. This article should not be taken as me being salty over any decision at a recent show, or upset that any show doesn’t use my favourite judging system. That is not the case, and I’ve intentionally saved this for publication at an appropriate time in between shows so it doesn’t read like sour grapes or criticizing any particular show organizers.

Second, I think at the end of the day, it is important to keep things in perspective. Everyone who is proud enough of their work to show it off is a winner, and the true prizes are the friends you make along the way. While competitions can be fun, getting too competitive about your hobby is a surefire road to misery and frustration (see: why I don’t do competitive Warmachine anymore).

Judging systems

The first, and probably the most established, judging system is IPMS style 1-2-3 judging. Here, there are multiple categories and a team of judges simply chooses the best, 2nd best, and 3rd best model on the table. While this may sound subjective, judges are trained to judge models according to standards laid out in the judging document. In these standards, craftsmanship is king. They have wisely chosen to avoid judging historical accuracy to prevent fistfights over the proper shade of RLM 69 or the number of rivets on the glacis plate on the real thing. And since they’re focused mainly on craftsmanship, they tend to be fairly objective – a missed seam line is a missed seam line, regardless of a judge’s level of affinity for the subject.

We also have the open system which is common in figure shows. Instead of competing against each other, modellers are competing against a set of objective criteria. There are generally Gold, Silver, and Bronze medals, and each entrant simply gets what the judges think their work deserves. This means that there can be any number of people winning any colour of medal – and zero is still a valid number.

Next, we have systems that are based on rubrics. This is common for Gunpla builders, and I believe it is often used for AMPS shows and paint scores in 40K tournaments, but I’ve never been to either of those. Here, there are a number of categories representing various aspects of a build – construction, painting, modifications, etc. Points are deducted for mistakes and added for things that are particularly well done, and at the end of the day, points are added up across all the categories and all the judges, generating a total score. This score can be then used either to rank the models most points to least, or give out awards to anyone who scores a certain number of point.

Finally, we have a simple people’s choice award, where attendees vote on their favourites and he who gets the most votes wins. The voter base can either be fellow entrants or the general public.

Of course, the differences between these systems aren’t always set in stone. Figure contests regularly feature best in show awards in addition to the gold-silver-bronze system. IPMS shows may use the open system for junior categories, so as not to crush the dreams of any young modeller. An open system can judge every model individually, or a modeller’s work as a whole, only giving out one award per category for his best work. And any show may incorporate a “people’s choice” award in addition to the judged criteria. Further, there are variations in these – IPMS contests can have “no sweeps” rules to prevent one person from taking home all the prizes in the categories, GSB style contests can judge and award every single model or an artist’s work as a whole, and rubrics can be either extremely detailed or very basic and can weigh different aspects of a build.

My thoughts

All of these systems have their pros and cons, and they are all heavily ingrained in the culture of the communities that have adopted them.

The IPMS style is nice and simple, and you can get a lot of judging done quickly. A simple glance over the table can greatly narrow down the number of models that are in contention and that a judge really needs to examine closely. However as anyone who has tried to organize a model show can attest, when you have very diverse builds, the sheer number of categories required to ensure that you have like competing against like, that there are not too many or too few entries per category, and that every possible thing that can show up has a category, ends up becoming a bit of a nightmare. Not to mention when the number of entries in each category is invariably much different than what you planned, judges often have to split or merge categories for there to be any semblance of fair competition. And then how the judges tweaked the categories on the day of the contest can be a bone of contention for those who didn’t win.

Further, there will always be some categories that end up being more hotly contested than others, so in some ways it is less of an objective measure. You can win first place in your category either by doing an amazing model and beating the other couple dozen amazing 1:48 aircraft, or you could be the only entrant in an obscure category and take home first place with a mediocre entry. I know that personally, the amount of hardware that I bring home at an IPMS show within driving distance depends about as much on who else shows up as it does on my quality of work.

There is also a risk of disappointment for those who don’t place. Anyone from 4th to last has no idea how well they did. As one example, I judged a competition a few months ago where we narrowed it down to the final four, figured out the winner and runner-up fairly easily, and then spend a lot of time deliberating over who gets the bronze and who goes home with nothing. If I hadn’t talked to the guy who was in 4th place afterwards, he may have been discouraged simply because he would have no way of knowing how close he was to placing.

Finally, because the IPMS style is comparing the models to each other, there are some logistical challenges. You need to have all the models on the table before you start judging, whereas with points systems and open systems, if the judges know they are going to have their work cut out of them, they can simply judge models as they arrive or get a head start before the entry deadline.

Open System

The open system has a lot of advantages. The main one is that you aren’t competing against each other, you are competing against yourself. How you do on the day doesn’t depend on who else shows up. This gives you a more accurate representation of where your skills are at, and it can help you set realistic goals, such as if you won Bronze one year, to go for Silver the next.

The open system also doesn’t require as many categories as the 1-2-3 system, which makes things a lot simpler. However, the judges tend to have a bit more work cut out for them as under a 1-2-3 system, a quick glance can often knock out a majority of the entrants who are uncompetitive and cut down on the number of models that need a serious look. The open system requires the judges to look at every model on the table (or, at least, everyone’s best work in a category) and give it its appropriate award.

I also think the prevalence of the open system contributes to the friendliness and mostly drama-free nature of the miniature painting community. When you are very rarely directly competing against each other, you are more invested in helping each other build their skills. When your friend wins a gold medal, it’s not bittersweet because he beat you to get it.

Rubrics and points

As for rubrics, to be honest, I’m not a big fan for a few reasons. First, I feel like they are popular because they have the appearance of objectivity, however I’m not sure they are actually that much more objective than something like the IPMS style, where there are no rubrics and the judges just confer and make a call. The theory is that simply choosing the best is open to favouritism, and points systems make things more objective. However, there is nothing preventing a biased judge from simply giving their favourites more points than the rest, either consciously or unconsciously.

Second, rubrics encourage people to build to the rubric if they want to be successful in the competition. Instead of simply approaching the build how they want, people end up having to tick off a number of boxes to get the maximum score or at least a competitive score in the category. For example, competitions that award points for conversions and customization may encourage people to do unnecessary modifications just so they can say that they did a conversion and get the points for that category. If the rubric isn’t well-designed, then you could have situations where it discourages creativity by rewarding certain specific style choices – for example, punishing an automotive modeller who enters a showroom clean car by giving him a big fat zero in the weathering category.

Can the people be trusted?

People’s choice is an interesting one and it makes judging a lot easier by simply removing the need for judges altogether. That said, in model shows as in democracy, the people can’t always be trusted. People tend to be attracted to the biggest, shiniest model with the most blinking lights. However, said model may, on closer inspection, have a ton of seam lines, alignment issues, and problems with the finish which are not apparent to the layman at a quick glance. Good models can get passed over simply because they aren’t the biggest, most eye-catching thing on the table. However, just having a show with a ballot can make for a much more chilled out atmosphere than a judged contest.

Feedback?

While feedback is good, not everyone wants feedback and not all feedback is useful. If you missed a mold line, being told to clean up your mold lines doesn’t really help you as a modeller. You already know you’re supposed to deal with that; that’s not new information for you. However, feedback such as new weathering techniques, suggestions on diorama composition, colour theory and lighting, etc., can actually help bring you to the next level.

I also feel that when it comes to feedback, some people are genuinely interested in improving their skills and progressing as a modeller, and some people just want to be salty that they didn’t win. They don’t want honest critique, they want to know why they lost and use it as ammo to grumble.

Again, I feel like the GSB system lends itself more to honest feedback because judges can tell you what you can do to bring yourself to the next level, rather than comparing your entry to other models and saying “well, Jim over there made something nicer.” Knowing what you can do to go from Bronze to Silver is much more valuable information than knowing which mold line you missed or what tiny flaw in your model caused the other guy to beat you.

If you really want feedback, track down a judge and ask him or her for it. Or, just talk to your friends and fellow competitors and get their feedback. Detailed feedback on a score sheet sounds nice, but it is onerous for the judges and it may not be as useful as simply having a conversation about your work with friends, regardless of whether they have a fancy judge title or not.

Prize Support

This is a tricky subject. I mean, upon initial glance, prize support is good, right? Everyone loves free stuff, and the more expensive free stuff, the better. If we can get hundreds of dollars of stuff and give it to the winner, then that’s great and it will encourage more people to join in, right?

However, there are some issues with large prize pools that may outweigh the draw of free stuff. When you have prizes like free trips to Japan to compete in the GBWC world cup or the $10,000 Crystal Brush, the stakes get higher and competition starts to get more intense. You can see this in things like Magic and Warhammer tournaments with large prize pools; by raising the stakes, you encourage people to be more cutthroat about things and increase the risk of salt and unsportsmanlike behaviour. IPMS deals with this by insisting that the prizes don’t have much if any intrinsic value – instead of wads of cash or expensive uber-kits, you are rewarded with a small medal or trophy, bragging rights, and the warm fuzzy feelings you get from recognition by your peers.

Further, in a smaller community, if you have a small number of people who are winning all the time, it can start to feel a little discouraging for the rest of the modellers. If you have entry fees and the same people winning all the prizes all the time, it can start to feel like same guy is taking your lunch money every day and this can encourage people who don’t have a chance to not show up, not participate, and not learn and grow by competing.

I’m not saying that prize support isn’t appreciated or that free kits isn’t a nice thing to have, but if people want the prizes badly enough that it starts to negatively affect their sportsmanship and attitude towards the competition, it can be a double-edged sword. How much is too much, and if you do have a generous sponsor, what portion of the prize pool should go to reward performance in the competition versus participation prizes like raffles, silent auctions, and door prizes, is worth some careful consideration.

Conclusion

Unsurprisingly, given my background, I like the open system. However, I feel like at the end of the day, too much competition can be unhealthy and perhaps there is something to be said for doing exhibitions and pageants instead of competitions. Aside from the occasional outburst of drama due to disagreements with a judge’s decision, it is all too easy to fall into a trap of comparing oneself to others in a hobby that should be about relaxation and, if you want to push it, self-improvement. Regardless of the format of the competition, one would be well served to go into it caring more for the friends you make and the inspiration on the tables than the awards you win.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s