What do you think of the current rating system ?

C-Fan · Post by **C-Fan** » 02 Jun 2005 16:28

In the first couple Sick 1 and Sick 3 comps, people only voted for who they thought was #1. Now, in order to have your vote counted, you have to put your choices in order from #1 to #10, or however amny entries there are.

I understand the reasoning behind the initial proposal, which was to reward the effort of the people who submitted combos which were excellent, but clearly #2 or #3.

Frankly though, if I am to fairly rate the submissions from #1 to #10, that means I have to watch the video several more times than if I only had to vote for the top choice, or the top 3. This also usually comes down to me almost arbitrarily saying one guy`s osis was better than sombody else`s butterfly, and to be perfectly honest, they have the same impact on me.

I definitely have enough time to watch a video once or twice, and thats enough for me to determine what was clean, as well as what I think was the best, or even top 3 combos. But this current ranking system where you have to vote for all choices... I don`t have the time or motivation to do so.

I was wondering how other people feel about it, so I made this poll. Maybe the results will bear me out in the minority, but if other people feel the way I do, maybe we could readjust the voting system in future comps so you only vote for #1, or for top 3.

mosher · Post by **mosher** » 02 Jun 2005 17:07

I think a top three is optimal.

professor · Post by **professor** » 02 Jun 2005 20:20

I agree. That's why I haven't voted because I don't think I would be able to rate 10 tricks fairly, but the top 3-5 would be acceptable.

max · Post by **max** » 03 Jun 2005 01:22

Good point Ken. The actual system is better than the previous one, but as you pointed out it takes a lot more time and raking n°7 through n°10 can get a bit arbitrary.
I'd propose placing the top 5 finishes because as you may have noticed there are more and more people competing in these events. Top 3 seems too close to the first voting system (voting for best combo only). I think top 5 is a good middle ground.

good topic.

DoctorJay · Post by **DoctorJay** » 03 Jun 2005 01:36

is there an opportunity to do both?

I mean you can vote for the top 5 or the top 10 .

I think that persons who won't finish in the top 5 ( as me ) are a little bit demotivated to compete in general .
With the new scoring system I get my rating anywhere- quite nice for me!
On the other hand this kind of rating takes a lot of time so many people won't vote - quite bad for all.

Summing up there should be one option to vote for all like Jeremy and others did - props for that that really helps and it is one more factor to compete.
another option should be to vote only the top three/five so the people who just wanted to make their pics could also vote.

What do you think

Jeremy · Post by **Jeremy** » 03 Jun 2005 01:37

I don't really mind what system is used but I think it's probably ok if you rush over the voting - It's just an informal online competition - there's no actual prize and I don't think too many people will be devistated if they don't win. I don't take very long over my voting decisions and I don't worry too much about wondering if I got them wrong or not. Probably in hindsight I might change my results - but I don't see much significant difference coming from that.

BalinorNZ · Post by **BalinorNZ** » 03 Jun 2005 02:07

Interesting topic, you might notice the same thing was bought up before the new system was implemented, unfortunatly nobody spoke up then.

Currently the scoring system relies on the fact that each voter must vote for everyone. If we had a top5 or top3 system there would be several faults:

Players who are at the bottom of the ranking would not get a placing, and would be less motivated to participate again, the obvious way to combat this would be an intermediate leage but that brings along a whole barrage of new problems.

The scoreing system would disappear, you cant have accurate scores for the players unless all of them are voted for with the current system, you would just have rankings 1 to 5.

A system where you have the option of voting for as many as you like might work, but there would be no incentive for anyone to vote for all the players so the bottom may still go unranked, discouraging new players from entering, also a scoreing system would be impossible, only ranks could be calculated semi accurately.

The way I see it is, players put alot of effort into entering these events and its kind of inconsiderate not even being bothered to to watch their entries carefully enough to rank them.
This is simply a way for the comunity to feed into the comunity. People enter the events and others do them the service of voting and constructively commenting on the entrys.

There will be a new voting method coming out soon, it is currently being developed by Jon Haber. Basicly you go to a webpage, select a player from x number of dropdown boxes, one for each rank, and click submit. Would you find it such a task voting if this was the case?

If as you say this is far too time consuming for the average voter then something must be changed, we do need to look after our voters aswell as our compeditors, a balance must be found.

Perhaps we could trial a system(for June?) where you can vote for any number of players you wish and the votes will be pooled and an aproximate ranking will be determined with no exact score.

What are your thoughts?

crazydwarf · Post by **crazydwarf** » 03 Jun 2005 14:03

Players who are at the bottom of the ranking would not get a placing, and would be less motivated to participate again, the obvious way to combat this would be an intermediate leage but that brings along a whole barrage of new problems.

Yeh this is my main problem with the top three. There's no way my sick 3's are going to be in the top 3 for a while so it would basicly mean I have no way of knowing how well I did. I like this way because it's so much fun to get feedback from people. If we split it up into 2 or 3 groups I would be fine with only voting for top 3 or so.

bigdirtyfoot · Post by **bigdirtyfoot** » 03 Jun 2005 15:29

crazydwarf wrote:
Players who are at the bottom of the ranking would not get a placing, and would be less motivated to participate again, the obvious way to combat this would be an intermediate leage but that brings along a whole barrage of new problems.
Yeh this is my main problem with the top three. There's no way my sick 3's are going to be in the top 3 for a while so it would basicly mean I have no way of knowing how well I did. I like this way because it's so much fun to get feedback from people. If we split it up into 2 or 3 groups I would be fine with only voting for top 3 or so.

You took the words right out of my fingers. I think that everyone should receive a rating, but there should somehow be a choice for *cough*lazy*cough* voters who don't want to watch everyone. I think there should be different levels, or pools. Whatever, but I definitely think everyone should get a rating for sending in a trick (or three!).

Muffinman · Post by **Muffinman** » 03 Jun 2005 16:48

Sorry, I only read about half of the posts, but I didn't vote on the latest comps for this reason, also. I don't know -- maybe we're all just really lazy? It's such a burden, though. I enjoyed the videos SOOOO much and wanted to congratulate EVERYONE, but then when I said that you HAVE to rank EVERY SINGLE competitor, I was really turned off.
I think top 5 is good.
I think NOT ranking every single competitor could also be a GOOD thing, because then the competitors at the bottom of the ladder wouldn't feel as bad being labelled "LAST PLACE", or "SECOND LAST PLACE".
Well it's something to consider anyway. Maybe it could be politely requested to reward the efforts of all competitors by ranking as many as you feel compelled to?
All I know is that I completely avoided this forum after reading the "YOU MUST RANK EVERY SINGLE PLAYER" (or something) rule...

EDIT: I just read David's quoted reply -- mabye being placed last could be helpful? It could also be discouraging. maybe rather than RANK every player, we could be requested for more diverse FEEDBACK on the submissions. I know I feel more comfortable commenting on a trick or combo than giving somebody a "YOU WIN" vote. Maybe we just need to comment more on the players' clips who we find less impressive as constructive criticism?

crazydwarf · Post by **crazydwarf** » 03 Jun 2005 16:56

It could also be discouraging. maybe rather than RANK every player, we could be requested for more diverse FEEDBACK on the submissions.

Yeh, I'd be perfectly fine with not getting ranked as long as people commented on my tricks. It just feels like a waste when absolutly no feedback is given to you, good or bad

mosher · Post by **mosher** » 03 Jun 2005 18:08

Ooh!

Why don't we just let everyone give each person a SCORE instead of a RANKING!??

eg.

Kevin - 8
Dave - 5
Steve - 7
Randy - 8
etc.

then you can score as many as you want and the persons over all score will just be their sum points divided by the number of people that scored them.

YES!

This solves the lazy person thing I think.

professor · Post by **professor** » 03 Jun 2005 19:01

Mosher, that is pure genius.

I vote for his idea.

bigdirtyfoot · Post by **bigdirtyfoot** » 03 Jun 2005 19:41

mosher wrote:Dave - 5

How come Dave has to get the lowest score?

Haha, I think this is a good idea as well. Comments should still be allowed, of course. So... a scale of 1-10 score would be good.

BalinorNZ · Post by **BalinorNZ** » 03 Jun 2005 19:50

As usual your brilliant Tom.
I think that your point system would work pretty similar to the current rank system but you can give each player any amount of points u want (not just 1,2,3,4,5 etc) and u can give 2 players the same score and u dont have to give all players a score.

However, if the overall score is to be accurate all players would have to be scored in each vote (imagine if everyone just put jorden - 10, Tom - 5) and no body else was voted for? Jord after 10 votes would have 100, Tom would have 50, everyone else would have flat 0, does that mean they are all equally 10x as crap as Jord and 5x as crap as Tom? No it means nothing really.

You could however form a ranking out of the scores (provided atleast one person gives a score to each player).

If anyone really thinks that going back to divisions has some validity then we could talk about that too.

Can anyone see a problem with tom's point system?

HG · Post by HG » 04 Jun 2005 04:01

Yes go back to divisions. Its painful to watch Inspinning pdx blender, irish cream, symp quantum atomic ss mirage, and then a blur.... or a paradon. The Divisions were great. And we know that some people have no chance of even getting close to winning if there is just one huge division (me)

B_Man · Post by **B_Man** » 04 Jun 2005 10:37

What's good about this type of voting is that everyone gets to see their placement past first or third place, which gives them a better idea of how they place and who they are competing against. This is the case if you limited the number down, the skilled players aren't affected but it's the ones on the bottom that do.

If we break it into divisions, Open and Intermediate, like last time then that also presents the problems of people being apathetic towards the Intermediates again - it was a bit clear that people only really wanted to see the big tricks links, not tripless stuff or whatever. In this case, they still get less attention

I don't know what we can do to help solve this problem, personally I like seeing how everyone places against each other, but the time it would take to vote really does throw me off. The number system sounds a bit like the system we have now except it seems like a lazy way of doing it. I don't know, I just would like to see everyone being judged fairly regardless of their skill, ya know?

BalinorNZ · Post by **BalinorNZ** » 04 Jun 2005 17:10

Thanks you Byrin, you said exactly what is in my head.
Still I think a new system may be needed so this must be discussed further.

C-Fan · Post by **C-Fan** » 05 Jun 2005 23:47

Some thoughts:

I think the apathy towards intermediate entries was resolved when Mosher started putting all the entries in one big video. By doing so, it forces everybody to watch all the entries. Therefore, I think having intermediate divisions could be cool.

As for the whole problem of those people not in the Top 3 getting no feedback... that is a legit concern. Whenever I voted in the past, I always wrote a somewhat detailed explanation of why I voted the way I did, and it addressed everybody`s entry.

I also have a slight issue with the word "lazy." Some of us have dayjobs, and when i`m off work, I`m not going to waste my time on modified watching vids. That`s something I sneak into work (OK, so I`m "lazy" by work standards

), and even then I don`t have enough time to watch a video several times over. It boils down to an issue of honesty. If I wanted to, I could rank the top 3 or top 5 after 2 or 3 watchings of the vid, and then just randomly assign values for the remaining entrants. BUt that wouldn`t be fair to them. If I wanted to do an absolutely fair ranking of all the entries, I`d have to watch the video a good 10 times or so. Some people just don`t have that kind of time, and if it comes down to me Bullsh*tting the rankings of half of the competitors, or not voting at all, thats an easy choice for me.

Another voting option is to have people vote for the top 3 Open combos, and then vote for the top 2 intermediate combos.

Senor Grommet · Post by **Senor Grommet** » 06 Jun 2005 09:57

Im confused as to why it would take someone 10 viewings to rank 10-15 players. I believe I could perform this task within 2-4 viewings, which doesnt seem that much based on the work that was put into the vid inthe first place by its makers and trick performers.

1 read the tricks performed beforehand
2 watch for cleanliness
3 watch for style, control, ease of performance
4 rate difficulty of tricks in own mind
5 make ratings accordingly

simple.

What do you think of the current rating system ?

What voting system would you prefer?

What do you think of the current rating system ?