One of the most confusing things about competition, especially ISI competition, can be "why didn't the best skater win." So I've created a sample scoring sheet to explain how this might happen.
Please note that I have "rigged" this sheet-- it is not based on any real skater, I made it up, sitting at my desk, to illustrate various points. Randy Winship, Director of Skating Programs at the Ice Skating Institute, who reviewed this for me, strongly recommends that you read the competition manual for judges (someone let me know if you can open this; it may be allowing me in because of my professional membership) which is available as a free pdf download on the ISI site (search "Judging Essentials").
ISI competitions use a panel consisting of 3 judges, one of whom must be Gold tested, and a referee, who I believe must be Gold tested, all through the ISI judges certification system. Each judge has their own sheet like the one below (I've consolidated all three onto a single form for ease of discussion). They mark only their own set of columns. Judges do not discuss marks except in certain instances. It is impossible for a single judge to rig an outcome.
PLEASE NOTE THIS IS AN EXAMPLE ONLY AND NOT IN ANY WAY CONNECTED TO ANY ACTUAL EVENT!
So who wins this event, and who was the best skater? Looking at the individual skills marks, looks like Skater 3 is the best skater, so how come she came in fourth? The worst skater- Skater 1, pulls off a third.
What happened?
ISI gives the judges a statutory range, based on the number of skaters in a flight, which is not supposed to go over 5. (For scheduling and other reasons, competitions may stretch this to 6 or even 7; the largest ISI flight I've ever seen was 9.) You'll see at the left top of the sheet "7.0-7.8" which is the statutory marks range for a flight of 5. Judges must stay within this range, except in certain prescribed instances, and must use the entire range for each skill. Judges grade ONLY the skills and qualities listed on their sheet. They do not see, or discuss, what the other judges are doing, unless there is a dispute or a question.
Looking over the sheet, you see everything within the marks range, except in the Back O-I Pivot and Duration columns (Judge 1), and the Change Foot column (Judge 2).
A correct duration gets 10 points (the referee has a stopwatch), but if you go over time (more than 10 seconds from start of movement to end of movement) there are prescribed penalty marks. Skater 4 went between 10 and 15 seconds over her time limit, so she loses 2 points. The referee notes this on her sheet, and initials the judge's sheet. Some coaches will mess around with the time limit; knowing that you've really got 10 extra seconds, they'll cut a 90 second program for 99 seconds. If the equipment plays that music a little slow (this happens less with digital recordings than it used to with audio tapes), or if she skates slow and finishes after the music ends, that skater is risking a penalty.
Silly skater 3 didn't do a pivot, so she gets a goose egg for the skill. This is not a bad pivot or an incomplete pivot, but a missing pivot. When this happens, the judge who is marking this skill will ask "Was there a pivot?" If all the judges and ref agree there was no pivot, it goes down on the judge's sheet and the ref's sheet as a zero, and the ref must initial the judge's sheet.
Skater 5 messed up her change foot spin. She might have had insufficient rotation, or left off the second forward spin. A recognizable attempt at a skill that is not completed gets a penalty mark of 5. This is NOT the same thing as a fall, or a badly executed skill. A fall on a completed skill will just put you at the bottom of the marks range (assuming someone else isn't even worse) and will cost you placement under "correctness" and "general overall"; falls on completed skills in ISI are not specifically penalized unless it prevents the skater from completing the skill. When there is any question at all, every panel I've ever been on will put a disputed skill at the bottom of the marks rather than giving a penalty mark. ISI judges try their hardest not to give penalty marks.
Again, all penalties must be agreed upon by the entire panel, and initialed by the ref. Some rinks also complete "penalty sheets" to back up any penalty calls in case of disputes.
And this is how it happens: Skater 3 had the highest marks across the board, and didn't do a pivot. Maybe she just forgot it. Maybe her coach is an idiot and didn't choreograph it. Notice that this error also cost her on Correctness and General Overall.
Skater 1 loaded her program with tons of extra content and apparently had great choreography, because her Pattern mark is also high, knocking her to the top of the heap with all her extras. That's a smart coach, and a good choreographer, focusing on this skater's strength.
Skater 5, also a good skater, needs to fix her pivot. But she's feeling pretty good right now, because she doesn't usually beat Skater 3.
Skater 3's parents are standing at the Skating Director's door, screaming at her. After she shows them the video, demonstrating that their kid missed an element, they're going to scream at poor Skater 3 all the way home for forgetting her pivot, and then fire the coach, even if it wasn't her fault.
Skater 1, who always comes in last, even against the book, as well as her parents and her coach, having no idea that all this is going on, now believe that she has a shot at the Olympics.
Posts
Well, this is a really bad system, in my view. Here is why: setting a range of 7.0-7.8 for each element but giving a score of 0 for missed element means that the difference between a very well performed element and a very poorly performed element is 0.8, but the difference between a very poorly performed element and a missed element is 7.0, almost 9 times bigger!
ReplyDeleteThink about this: what is better, do 8 elements perfectly and miss 1 completely, or do all 9 very poorly? I think the first should be better, but according to the above system the second competitor will win (the first will receive 8*7.8=62.4, and the second will receive 9*7=63).
Maria, mom of 2 skaters: FreeSkate 2 and Snow Plow Sam
I also recently realized that there is another factor that influences the outcome of a competition but actually has little to do with how good your skater is: namely, where the skater is at the given moment, relative to the competition levels. May be it is true more for USFSA than for ISI competitions. What I mean is that the competition levels (in USFSA) are pretty far apart. It often takes a year to progress to the next level. This past summer we had a dilemma about where to put my daughter: she grew out of No Test Level 1 (swizzles, wiggles, etc., no spins, nu jumps allowed - she had a good waltz jump and 1 foot spin, was working on more jumps and spins), but she was either still working or not even working on some elements you see in No Test Level 2 (they were not required but just allowed; but of course the top competitors were doing everything that is allowed). We decided that Level 1 was too low and too boring for her and put her into Level 2, with understanding that she'll just try to do her best but she had absolutely no chance of winning just because she was not there yet. She will be there in half a year, but just not yet. Of course, she came in last. But this doesn't mean that she is a bad skater! This means that the levels are too far apart, and it was impossible to find a good fit for her.
ReplyDeleteWhy did we go to this particular competition, you may ask? To make the long story short, we live in the middle of nowhere, and we don't have many competitions here, but we were near this competition during our summer vacation, so we decided to go there.
Maria, mom of 2 skaters: FreeSkate 2 and Snow Plow Sam
The 0 penalty for an element that isn't performed seems harsh, but how else are you going to do it? This is where proper coaching is important. In a system where you are judged on required elements, you cannot grade an element that isn't performed. Trust me, it KILLS the judges when a good skater places poorly because of this, but it's not like people do not know what the system is. And judges will bend over backwards to avoid giving that penalty mark. You wouldn't believe some of the things panels I've been on have agreed are pivots, just to avoid that penalty mark.
ReplyDeleteI agree that a missing element should receive 0, but setting a range of 7-7.8 is a bad idea, IMHO. I think each element should be judged on a scale from 0 to... whatever. Say, from 0 to 10. Then:
ReplyDelete0 = missing
1 or 2 = barely attempted
3 or 4 = poor
6 or 7 = good
10 = perfect
something like this, where the difference between "missing" and "bad" is comparable to the difference between "bad" and "good". The first difference could still larger than the second, but not 9 times larger!
Maria, mom of 2 skaters: FreeSkate 2 and SnowPlow Sam
The narrow judging range makes it much harder for a single judge to manipulate an outcome, since on a 3-judge panel you can't just delete high and low, like they do on larger panels. If a judge could just put a skater at the bottom of a 10-point range, then there would be more judging shenanigans.
ReplyDeleteThese rules and standards are not designed to make it perfect. They are designed to keep the system as fair and even as possible.
The 0s are harsh. But I can tell you, when skater grrrl got to Juvie level and got her first 0 because she recentered a combo spin, it was a lesson she never forgot.
ReplyDeleteAnd it happens at the top levels, as well. I've seen it enough times at elite levels with a botched spin or something wonky in the footwork and an entire score is lost. It's heart-breaking, but there are no consolation prizes for elements not done with the IJS system.
Just ask anyone who ever fell on a required jump during the short program at the Olympics.
ReplyDeleteWhile ISI does make it as painless as possible, in the end, if you do not satisfy the rules and conditions of the competition, it costs you.
Because of the judging system, we choose not to use ISI as a replacement for USFS competitions. We use them as a fun supplement to USFS competitions. We will never again (after 2 unbelievable outcomes) enter a Freestyle # or Bronze/Silver/Gold/Platinum freeskate event. We only enter Spotlight or fun events like Interpretive. We are lucky because we live in an area where there are many local competitions to choose from, both ISI and USFS. Our family enjoys the ISI competitions for their fun and creativity.
ReplyDeleteWe know skaters who regularly enter ISI competitions, and consistently do well, but always finish poorly at USFS competitions. Some have decided not to enter another USFS competition because they think the judging is too harsh. I feel the judging is more consistent and valid for a competitive skater. It's important to keep in mind that ISI exists to promote recreational skating, therefore the fun events that the kids enjoy. USFS is all about competition and the judging system reflects that.
This is an incredibly enlightening post. Thank you.
ReplyDeleteThanks for putting up the score sheets with an explanation like that.
ReplyDeleteNow I understand why my daughter's coach puts each required element twice in her USFS basic skills program (missed one try and the other can still count). And also always follows a required element with a bit of choreography - so if my daughter falls there is time to get up - skip the choreography part and get back into her program at the required element.
We also have in mental checkpoints along the routine where if she hears she is running behind she can ditch something (say skip the "falling leaf") and just go to the next thing so she will end on time. We always have a plan A (everything goes as planned), plan B (if you run behind) and Plan C (if you get to the end and need to do a little jig before you pose if the music still has a little left to go).
A lot of this just comes down to smart coaching.
"We always have a plan A (everything goes as planned), plan B (if you run behind) and Plan C (if you get to the end and need to do a little jig before you pose if the music still has a little left to go)."
ReplyDeleteLOVE this! And yes, this sport really encourages kids to think on their feet. (Literally and figuratively, lol.) Now, in upper levels, you can't just toss in extra jumps the way you used to be able to in the 6.0 system (Midori Ito famously tossed in a 3Axel after she botched the first one) but yes, listening to the music and improvising within the rules is something that good skaters do all the time.
It does take practice doing it in practice, though. Those kids that don't get right up from a fall during run throughs or who don't have a repertoire of "add-ins" aren't going to magically come up with them in competition. It took my daughter a long time to learn this. Once it was clearly put to her that she was given not one score, but 8 technical scores and that she could still place well after botching one or even two, she had an "Ah HA" moment. Now she fights for every element, even if she landed on her tush the one before. :-)
Well, to be clear, I know most coaches don't encourage improvising if there isn't a need. If skater grrrl does mess up but successfully covers it, she never gets congratulated by the coach. However, she doesn't get yelled at, either. (Usually. As long as she's been working hard up to competition date. If she's been slacking off AND has a lot of boo-boos, we don't do the "hug it all away" approach.)
ReplyDeleteThe goal is to do the program you're supposed to skate. However, in a worst case scenario - and everyone will have them - a program can be salvaged.