Rant about "how to judge a routine"

General footbag-related topics that don't fit elsewhere go in here.
User avatar
DoctorJay
Fearless
Posts: 540
Joined: 20 Sep 2002 05:46
Location: bielefeld/germany
Contact:

Rant about "how to judge a routine"

Post by DoctorJay »

This is copied out of my blog . I wanted to point out the problems in judging routines in general .
All happened at German Champs 2005



My rant about judging a routine !!!!!!:

The judges had to give one number for technical merit and one for artistic merit.
I am quite sure that the judges had no idea about the guidelines what things refer to tm or am.

I had a small discussion with one guy who participated in euros worlds and in a bunch of other big tourneys.
He said that in his view a drop counts in both categories because it lowers your tm and your am ?!
In my opinion tm is just based on the techincal difficulty of a routine and nothing else - only the choice of the tricks and their links nothing else--> the rest is artistic merit.

Of course we had different opinions on that issue but if this is not clear for all the judges the judging doesn't make sense because everybody interpret other things in the words "technical merit" and "artistic" .

I could not find any official guideline for the clarifying of tm and am but it is necessary to get going with routines .
Moreover you always hear that later routines have advantages to earlier routines in a pool- in my opinion that is not a good sign for the qualification of a judge.

in a pool less than 8 contenders you should be able to make some notes after each routine and so remembering each one should be not that hard .

I would feel uncomfortable to judge when I am not prepaired enough. Every judge should consider how much time he/she spent before in getting some judging skills .

Being a good player yourself does not make you a good judge.

A good clear judging guideline or much better a judging certificate are important to honor those people who elaborate their routines .

Otherwise I cannot see the sense to prepair a routine and right after it gets underranked because the lack of knowledge of the judges.

Summing up this is not a rant that I wanted to get placed higher , I am satisfied with my 7th place but this is not the way a professional judging should look like - My numbers in am varied from 3rd place to 7th - this should not be possible .
Jochen Bauer

Paradox Feet Footbag Club
http://www.paradoxfeet.de

Website:
http://www.jochen-bauer.net
User avatar
Jeremy
"Really unneccesary"
Posts: 10178
Joined: 08 Jan 2003 00:20
Location: Tasmania

Post by Jeremy »

Yeah I understand how you feel. I guess it's a fairly big issue that needs to be dealt with. I can't really say how I personally feel about judging but I can say that there is discussion in the IFC about it so maybe things will improve.

I think it's important to remember that footbag is a very new sport and as such many people believe that we shouldn't have the same expectations as what we'd have with older sports. Personally that's not my view but I'm sure you can understand that position.
User avatar
sergio
Multidex Master
Posts: 265
Joined: 14 Sep 2004 09:33
Location: Paris

Re: Rant about "how to judge a routine"

Post by sergio »

DoctorJay wrote: I had a small discussion with one guy who participated in euros worlds and in a bunch of other big tourneys.
He said that in his view a drop counts in both categories because it lowers your tm and your am ?!
In my opinion tm is just based on the techincal difficulty of a routine and nothing else - only the choice of the tricks and their links nothing else--> the rest is artistic merit.
In my opinion, that guy is not completely wrong, but only to a certain extent. If a player performs a routine with 3 drops, the 3 of them on the only 3 7 adds he tried to hit, I don't think I'd put him the same TM as I could have if he hit 2 of them and dropped on 1 other random trick.

I don't think it is possible to rationnally define what goes in AM and what goes in TM. The same goes for diversity and bothsidedness. Would you put that in AM or TM? May be that's my footbag subconscience speaking, but I definitely find a routine to be more artistic when there is a lot of diversity and when tricks or done both sides, and I also find a routine to be more difficult when there is a lot of diversity and when tricks are done both sides.

May be we should do like in gymnastics. Put 7 or 8 judges and cancel the hishest and lowest marks on both AM and TM....
Serge Kaldany
75 Adds - Paris
User avatar
james_dean
space cowboy
Posts: 2268
Joined: 26 Oct 2004 23:11
Location: Bendigo, Vic, Australia

Post by james_dean »

yeah i've thought of that too (the gymnast system) but I think I agree with jeremy's sentiment that (and i'm only going by memory here so i probly wont get it completely right) variety, both sidedness, all that, is TM. Cause it requires *technical* skill, you can do a difficult, bothsided, hugely varied run, completely unartistically. Artistic ability is playing to the music, both beat and climaxes etc, playing to the crowd, links or rewinds that look funky... there were other artistic things but i can't remember them offhand. And the artistic bit there is in the order of tricks not the difficulty... so playing to the crowd on an around the world gets the same AM as playing to the crowd on PS whirl.

*is that right jeremy? well that's how I feel about it anyway*

Basically I definately think that variety and bothsidedness are technical merit. As for drops, i'm not sure. At first they seem completely TM but it definately hurts the presentation if you drop... so I don't know.


Oh yeah, using 7 to 8 judges and cutting off the highest and lowest still requires AM and TM to be properly defined...
Image

"It's a punk one!" - Auntie Val, after being shown a spikey footbag

Bloggy

Challenge
User avatar
Jeremy
"Really unneccesary"
Posts: 10178
Joined: 08 Jan 2003 00:20
Location: Tasmania

Post by Jeremy »

in my opinion:

Technical merit: How skilled the person is at freestyle footbag.
Artistic merit: How skilled the person is at performing.

So variety, bothsidedness, difficulty of combos and single tricks etc. is all techincal merit.

Playing to the music, reacting with the crowd, particular choreography, etc. is all artistic.

Unusual surface moves, flyers etc. have no extra artistic merit unless they are somehow used as part of the performance. Simply putting some flyers in a routine is a very low level of artistic merit and not a great deal of technical merit. However if they're a strong part of the performance (ie. on particular parts of the music etc.) then they obviously have a great deal more technical merit. Move selection in general is not part of artistic merit - nemesis has as much artistic merit as legover - it's how the moves are used in the performance that is key to artistic merit - although of course nemesis is a very technical move.

Regarding drops - drops show that the player does not have either the focus or the physical skill to hit that particular move consistantly - thus lessening the technical score. They also detract from the performance - thus lose artistic score. How significantly they effect the score depends on the amount of drops. I would say that the more drops a player has the less each drop effects the score. So the difference between 0 drops and 1 drop is a lot greater than the difference between 4 drops and 5 drops. It really depends on the routine as well and the nature of the drops.

edit: my opinions for this is based mainly on reading up on how pretty much all other freestyle based sports such as gymnastics, figure skating, etc. are judged. As well as trying to read as much as I can about what was intended with the currently used (but unofficial) "olympic" judging system.
User avatar
Outsider
Ayatollah of Rock n' Rollah
Posts: 1373
Joined: 21 May 2003 21:30
Location: Bridgewater, New Jersey

Post by Outsider »

I like Jeremy's explanation very much. Well said, O'WheelJ. There was only one little line from your last post that I'd take exception too:

J O'W said:
Simply putting some flyers in a routine is a very low level of artistic merit and not a great deal of technical merit.


I happen to think that flyers, if done well, can (not automatically and necesarily, but CAN) show a great deal of technical merit. Here is why: there is more to freestyle than stalls. "Dex-dex-stall > dex-dex-stall > repeat" is all well and good, but thats only one kind of technical play. Demonstrating ability to hit good, controlled jumping kicks (especially from and to stalling moves) shows mastery of more than one type of technical play. Flyers are pretty hard, actually. Come on, admit it, most of you out there can do ten times as many guiltless stalls in a row as guiltless flyers. Mastering flyers is hard and the proof is in the records: the record for Ripwalk, a 4-ADD trick, is almost double (95 - Allan Hagget) the record for the 2-ADD flying clipper (50 - Tms Tyrpeki), and people have been doing flying clippers since way before the ripwalk was ever invented, and man, that flying clipper record is way more than I've ever done in a row, and I do practice them. I don't remember how many Infinities (alternating butterfly stalls) Matt Cross said he had done just the other day, but the previous record for them was more than 8 times as great as the record for consecutive alternating Butterfliers (jumping butterfly kicks). Ben Skaggs from Las Vegas recently hit 31 spinning butterfly stalls (nicely done, Ben!), and that is more than four times the record for consecutive spinning jumping butterfly kicks. And transitioning from stalls to flyers back to stalls can also be pretty tricky. Throwing in some flyers CAN show some significant technical merit. Of course, they'd have to be done with good form, and so many people do their fliers with such atrocious form its almost painful to watch...

To sum up, I'd say that if your routine has nothing but stalling tricks, you might just be demonstrating that your technical ability is LIMITED to stalls alone.
"The time has come to convert the unbelievers..."

Jonathan Schneider --- sometimes showers with his Lavers on (to clean them)
The Ministry of Silly Walks
NYFA
BAP
hacksterbator
Washed-Up Child Star
Posts: 4141
Joined: 12 Jul 2003 18:33
Location: Winterpeg, Manisnowba
Contact:

Post by hacksterbator »

okay, let's get this clear once and for all...

if you've heard of or even slightly recognize the name of the person you are judging (you have to account for mispronunciation incase they are polish or czech, or somthing), it's normal to add on 2 or 3 extra marks once you've seen his routine. so if you saw it and thought it was a 3.0, 2.5 bump it up to a 5.0, 3.5 or some combination like that.

of course, if you have never heard of him/ her before, judge them accordingly. how ever, should their routine score higher that that of the people you have heard of, alter it accordingly so as to be out of contention with any names.

if the preson is representing country that you did not even know had a footbag club, score them, then dock them 1 point for not having a large enough scene where they are from.
A.G.
User avatar
shreddaily
Flower Child
Posts: 1665
Joined: 03 Sep 2003 16:56

Post by shreddaily »

nice t grant!

judging is a tangled web of confusion that is imprefect by definition. I've gone away from more than one tourney not thrilled with the judging, but the more I look at it most of the time it just meant the competition was realy good.
I have no idea how to judge routines better...I'm goning to judge this year at Funts instead of compete and see how the other half feels and maybe I'll have some more ideas/perspective.
Last edited by shreddaily on 29 Aug 2005 10:14, edited 1 time in total.
User avatar
sergio
Multidex Master
Posts: 265
Joined: 14 Sep 2004 09:33
Location: Paris

Post by sergio »

Jeremy wrote:in my opinion:

Technical merit: How skilled the person is at freestyle footbag.
Artistic merit: How skilled the person is at performing.

So variety, bothsidedness, difficulty of combos and single tricks etc. is all techincal merit.
So if 2 players play both really well to the music, react to the crowd, have a great style, blablabla ; but one does ss butterflies during the whole routine and the other only does uniques, would you give them the same AM? I mean if the difference between bothsidedness and variety of both players would be kind of limited, then I also feel it should only influence the technical merit, but if the difference is considerable...
To give you a realistic exemple, I personnaly would have upped Scott's AM a bit for his final routine of Worlds 2005, because he had like twice the number of uniques as anybody else, and I feel that it made his routine more artistic in a way than if he hadn't have that many different tricks.
Serge Kaldany
75 Adds - Paris
pillar
Shredaholic
Posts: 135
Joined: 29 Aug 2003 15:09
Location: Helsinki, Finland

Post by pillar »

May be we should do like in gymnastics. Put 7 or 8 judges and cancel the hishest and lowest marks on both AM and TM....
In order to accomplish this we would have to change the current judging system. Right now the scores don't really matter. Rank does.

I personally feel that right now, there are so huge differences at judges scores, that it would be unwise to move away from ranking based judging. Even if you cancel out the highest and lowest. Sports that use this kind of judging have usually very minor differences inbetween the judges anyway. Also requiring more judges would make it a little bit harder, but that's not that big.

We just need a lot more experience with judging, people interested at doing it and training others as well. Judging being as underrated job as it is right now, it will continue to have problems.

Online training could be possible too. Like having all the competition final routines available and asking people to judge them - submit the scores with a web form (not in forum, because then people would see others scores and it would affect the outcome). Then just gather all the information and post it here and have experts explain what would have been the right score and why. That way people would actually learn. Well, just an idea. Any thoughts?
Juha Linnanen
User avatar
sergio
Multidex Master
Posts: 265
Joined: 14 Sep 2004 09:33
Location: Paris

Post by sergio »

Cancelling the highest and lowest ranks is possible since you sum them all up at the end!
Serge Kaldany
75 Adds - Paris
User avatar
Jeremy
"Really unneccesary"
Posts: 10178
Joined: 08 Jan 2003 00:20
Location: Tasmania

Post by Jeremy »

I agree to some extent with Jon about flyers - I didn't word that very well. I think it's definitly possible to hit flyers with a high degree of technical merit. However I think that most of the time when you see flyers in routines they don't have a great deal of technical merit - clipper stall>ss flying clipper kick>clipper stall is not a very technical combo. There are definitely flying links that could be done with a great deal of technical merit. I was referring to what we usually see in routines - which is a short easy flyer section that usually gets the comment "this routine had artisitic merit because it had flyers in it" or something silly in that nature.

I really think though that this intire topic is pretty much a waste of time. How many people - upon reading this topic are actually going to try and improve things - ie. apply to join the IFC or another IFPA related committee/group and actually make a difference? Opinions don't help the sport at all - action does and pretty much all the opinions are out there. The big problem with footbag at the moment is not ideas or thinking - it's not enough people prepared to take the action.

Somebody mentioned online training for judges. This has been suggested to the IFPA Board of Directors some time ago now. Maybe it will go ahead - and we'll have an official judging committee - I don't know yet. There were some responces that were sort of positive but there's a lot of other different stuff going on and I'm the person who was driving the idea and I'm definitly not as committed as I was when I suggested the idea. I don't feel comfortable talking too much about it but I do think that people who love footbag and are committed to the sport need to start looking at doing more for it - especially at a high level.

I do countless demos and try to expose the sport as much as possible but that's a very small and fairly insignificant effort in building footbag into a strong professional sport compared with what needs to happen.
User avatar
sergio
Multidex Master
Posts: 265
Joined: 14 Sep 2004 09:33
Location: Paris

Post by sergio »

Haha, way to kill the topic Jeremy! :D

This topic is not that useless, at least the people who read it will put more thought in the different ways to judge a routine.. That's a start! :P

And there are many ways to help the sport. Creating clubs (federating footbag), getting new players to play footbag, hosting annual tournaments are also ways to help the sport.
Also, everybody doesn't have the time to get involved in everything related to developping footbag, and everybody isn't interested in that kind of thing and just wants to have fun playing!
Serge Kaldany
75 Adds - Paris
User avatar
DoctorJay
Fearless
Posts: 540
Joined: 20 Sep 2002 05:46
Location: bielefeld/germany
Contact:

Post by DoctorJay »

Regarding the situation of people who are willing to help:

I sent an email to join the ifc judging and seeding board ( before german masters) got two answers one of you Jeremy another one where my name was mixed with another person and now I never heard anything about it . Maybe something will change when the IFPA Forums will get going .... having no visible platform who is doing what and when --- nothing will change fast .

Waiting for replies is the most demotivating thing in the world .

Jeremy I am quite sure that there are people who are willing to help maybe it is not the appropriate surrounding .


On topic :

You are working on an official rulebook .

Is there a guideline how to judge a routine ?

If yes - where is it and where you can look it up ?

If not- let us make one ! Discussion is much easier when there is a finished product !

In my opinion the situation now is not satisfying so why don't we change it .

You are in this ifc commitee ( maybe I am too I do not know ) so who else should start to make it .

Thanks for your effort besides you may know that nobody will ever thank you .
Jochen Bauer

Paradox Feet Footbag Club
http://www.paradoxfeet.de

Website:
http://www.jochen-bauer.net
User avatar
Tsiangkun
Post Master General
Posts: 2855
Joined: 23 Feb 2003 02:27
Location: Oaktown

Post by Tsiangkun »

The IFPA seem completely uninterested in maintaining a dialog with players, and actually doing anything to help get rule changes implemented. Do you really think they will be motivated to help people judge an event they can't even be bothered to revise the rules for ?
User avatar
Jeremy
"Really unneccesary"
Posts: 10178
Joined: 08 Jan 2003 00:20
Location: Tasmania

Post by Jeremy »

I would disagree that they can't be bothered updating the rules. It's just that different people have different agendas and different priorities. I would say that the rule book will almost definitly be updated by worlds next year. Right now the priority is in printing a new edition of the rules. This edition will have the current official freestyle rules in it - ie. the rules that haven't been used since 2001. It would be inappriopriate for me to comment on the decision to spend IFPA money (that was aquired by selling previous editions of the rules book) on printing such a document but needless to say I've voiced my opinion in the appropriate place and this seems set to go ahead.

Regarding joining the IFC etc. It took me about 3 emails to get any kind of responce. My tip for anybody dealing with the IFPA is send an email every week until you get serious replies. "Thanks for the email Jeremy - I agree blah blah blah - I'll let you know how it goes" etc. is not a serious reply. Keep sending emails until you actually get some action.
User avatar
Jeremy
"Really unneccesary"
Posts: 10178
Joined: 08 Jan 2003 00:20
Location: Tasmania

Post by Jeremy »

sergio wrote:
Jeremy wrote:in my opinion:

Technical merit: How skilled the person is at freestyle footbag.
Artistic merit: How skilled the person is at performing.

So variety, bothsidedness, difficulty of combos and single tricks etc. is all techincal merit.
So if 2 players play both really well to the music, react to the crowd, have a great style, blablabla ; but one does ss butterflies during the whole routine and the other only does uniques, would you give them the same AM? I mean if the difference between bothsidedness and variety of both players would be kind of limited, then I also feel it should only influence the technical merit, but if the difference is considerable...
To give you a realistic exemple, I personnaly would have upped Scott's AM a bit for his final routine of Worlds 2005, because he had like twice the number of uniques as anybody else, and I feel that it made his routine more artistic in a way than if he hadn't have that many different tricks.
I think it's probaby very unlikely that somebody could perform an excellent routine only doing one move. - or even doing a small amount of moves. Hypothetically speaking if two people did routines which I thought were equal good performances then of course I would give them the same technical merit. Trick selection to music is obviously a part of artistic merit - doing the same trick over and over is not an exciting performance - it's boring. This has nothing to do with the difficulty of the tricks involved. If somebody did consecutive nemesis in perfect time with some music for a whole 2 minutes i would score them at less than 1 out of 6 for their artistic merit. I'd presumably give them a much higher technical score because 2 minutes of consecutve nemesis has a very high technical value. But this is a fairly pointless hypothetical question because I think it would be impossible to do an equally impressive performance with a limited amount of tricks. 2 minutes is a long time and seeing the same things in that time definitley gets boring.

Sorry for the double post.
User avatar
sergio
Multidex Master
Posts: 265
Joined: 14 Sep 2004 09:33
Location: Paris

Post by sergio »

The hypothetical question was just to expose my point. If you look 4 lines lower, there is a realistic exemple.
Serge Kaldany
75 Adds - Paris
User avatar
Jeremy
"Really unneccesary"
Posts: 10178
Joined: 08 Jan 2003 00:20
Location: Tasmania

Post by Jeremy »

So Scott's scores were:

technical merit:
5.0 5.4 4.2 5.2 5.0 5.2

Artistic merit:
3.6 4.9 4.1 4.8 5.0 5.3

I agree with the average for technical merit. (although not with the second or third score) - I think he showed a very good range of skills although a low amount of difficulty with only 1 drop - which I think is a perfect example of a low 5s score. 5.4 is definitly too low a score in my opinion for a low difficulty routine but 4.2 is also too low for such an excellent range of variety. If the top and bottom scores were discounted - as happens in a large amount of freestyle type events - the score would be the same as what I would have given. Not that the system works like that - it's on ranks but anyway - I think on technical merit the judging was exactly what you'd expect.

There is obviously a much larger range of variety amongst the artistic score - which shows that people had different interpretations of what they were judging. Even if you discount the top and bottom score you still have a range of .9 - which is a lot in a 6 point scoring system. Personally I would have scored him at about 4.5 - 4.8 - I think he played well to the music and put on a good performance but lacked any kind of build up or climax and also didn't appear to have any specific parts choreographed to the music. That is to say - there were no particular sections that stood out as working really well with the music. I think that really comes down to bad music selection - he chose music that really didn't allow him to put in anything that demonstrated that exactly how much choreography had gone into the rountine. Maybe every move was planned and pulled off exactly at the time he wanted in the routine - you really don't know - compared with the start of Vasek's routine where it is clear that those moves are hit in very specific timing with the rountine.

I think Scott's choreography was good but I also think that the kind of timing and choreography that he was missing is crucial for scoring 5 or more. I also think the lack of building to any kind of climax - ie. routine structure - is a very important thing in scoring over 5. Really the level of performance was the same through out his routine. It was still good - which is why I'd give a high 4s score. His average score from each judge was 4.7 so again I think that's a good score - although the large range of variety - 3.6 - 5.3 is way too big. That's 1.6 points difference or a 27% difference in score. Imagine if they were scores out of ten and one judge gave a perfect score and another gave 7.3.
User avatar
sergio
Multidex Master
Posts: 265
Joined: 14 Sep 2004 09:33
Location: Paris

Post by sergio »

I agree with you all the way. I actually didn't take a look at his scores before saying this. I just wanted to point out, that with everything being the same in his routine (drops, difficulty, music selection, play on the music, etc), I'd have put him a slightly lower AM with less uniques than what I would have put him with the number of uniques he actually did. Sorry for the bad wording!
Serge Kaldany
75 Adds - Paris
Post Reply