EDEB8 - Ultimate Online Debating
About Us   Debate    Judge   Forum

Important: judge feedback

< Return to subforum
admin
By admin | Jun 13 2015 10:23 PM
Edeb8 works on an honesty system. We basically expect people to be transparent and try to help one another, not put each other down. Recently, many of you will remember how I became so upset at how this wasn't happening in regards to judge feedback and hiding behind a 1-pointer, I had to leave the forums for a week. Now, the same user has had much the same problem (and I think we all know who it is).

Judges are expected to put in time and effort into producing high-quality judgments. It is only fair, therefore, that we expect judge feedback produced of the highest quality. Smug attitudes of "it's anonymous so I can say what I want with no repercussions" will not cut it, as RationalMadman found out. I deal with perceived abuses of the system harshly. Frankly I have no other choice. Positive feedback is an important incentive mechanism for judging. If not so judges can improve there is literally no reason for feedback.

At the time I wrote an important blog post for the site. I doubt anyone read it. But I hope people will. I take this site more seriously than I think a lot of you realize and I get extremely emotional when people disrespect the systems I've worked to put in place. Especially when that user is openly not following the rules for judgments and is rating lower than what the labels for the judgment state. Unfortunately this has been happening a lot in the last little while and I'll openly say, it's made me just a teeny bit depressed. And it has driven numerous others away from the site in the past few months.

So I'd like to have a serious discussion here. What can we do to ensure that every judge feedback is CONSTRUCTIVE for the judge, which is ultimately the intention. Options might include:
- Moderation ie banning offending users
- Features ie requiring some comment to go with the judgment rating
- Feature restriction ie making it non-anonymous (we'd obviously have to weigh that against, would that limit the incentives to provide such feedback?)

Tell me your views. Last time I made a thread like this, I was attacked as a moderator. When I tried talking to the member in question about this, they personally attacked the notion I should be a mod. That is most definitely the least helpful thing you can say here. Instead, let's focus on what I can do and the community can do, for the good of the site in general.
Thumbs up from:
I'm the main developer for the site. If you have any problems, ideas, questions or concerns please send me a message.
Let's revive the forums!
admin
By admin | Jun 13 2015 11:41 PM
admin: Bumping to the top
I'm the main developer for the site. If you have any problems, ideas, questions or concerns please send me a message.
Let's revive the forums!
nzlockie
By nzlockie | Jun 14 2015 12:22 AM
admin: Yeah it's awkward.
I'm not sure there's anything more you can do to fix the existing system where everyone gets a chance to give feedback. I don't think removing the anonymity would help.

The one thing that MIGHT work would be to remove the negative effects of a bad ranking on the judge's score. Assign points in a cumulative fashion rather than a percentage.
A better judgement gets more points but no level of judgement actually loses you points.

It's not a better system and it won't paint as accurate a picture as the current one does but it may remove some incentive from judge bombers.

I wouldn't be opposed to some moderation, but it gets a bit risky and I'm not sure it addresses the root of the problem.
Blackflag
By Blackflag | Jun 14 2015 2:09 AM
As long as anonymity is available for judges and judge ratings the use needs to be respected.
I am being attacked and harassed in multiple directions for something that I am allowed to do on the site, and that is ridiculous..

What's even more ridiculous is how the accusations and attacks on me rolled out before I even cast an anonymous rating. Seriously M8's, the reason I rate judgements is for the purpose of keeping the judges score fair rather than trying to give feedback.

Admin's arguments are getting sillier and sillier. First he says that it is moderation-worthy to not judge debates where the person you disagree with personally won the debate, but it is totally okay to just not judge debates. Now he is claiming that it is okay not to rate the quality of a judgement, but if you do rate the quality of the judgement you should be banned if you don't waste your time trying to give feedback coupled with that rating.

Alot of this is bullshit. This isn't the direction I want to see this site headed and the constant avocation of stupid moderation turns me off from the site.
Blackflag
By Blackflag | Jun 14 2015 2:13 AM
Here's one of my PM's with some minor paraphrasing.

Nope, not going to happen. The person in question has no way of knowing I rated his judgement, therefore owes me an apology for harassing and attacking me over something which he can' t even be sure I did*. IDGAF about how my judgements are rated, but this guy obviously does, or he wouldn't get all defensive the moment he notices someone rated his judgement badly.

I also don't have a moral duty to defend my justified rating. He should be grateful that he got his judgement rated at all. I don't have to go above and beyond here and explain my rating if I don't want to.

Now here's some advice for you. You're an administrator, not a site mediator. If you really feel these matters require delegation of your time, get someone else who can handle these things for you. I don't think it is healthy for the site when you put yourself in the middle of this silly drama.
Blackflag
By Blackflag | Jun 14 2015 2:21 AM
The one thing that MIGHT work would be to remove the negative effects of a bad ranking on the judge's score. Assign points in a cumulative fashion rather than a percentage.
A better judgement gets more points but no level of judgement actually loses you points.

On paper that sounds good, but there would also be some judges who constantly make poor judgements but are able to recieve a high judging score because every once in awhile they happen to be good.
admin
By admin | Jun 14 2015 1:16 PM
Blackflag: There is a reason certain features exist, and misuse of those features has always been grounds for moderation. This isn't necessarily an attack on your behavior, but on the position you have been advocating. I am a moderator first and an administrator second, whether you like it or not.

Judgments exist so that debaters can improve. Disincentives to this exist in the form of judge scores, but you've made it absolutely clear you don't care about judge scores. Therefore you have no incentive to improve the quality of your judging, and this is played out in one-point decisions you have been making despite a proven capability to do higher-point decisions, an obvious abuse of the intention behind that system.

In a similar fashion, judge ratings are supposed to make judges improve. There's a specific reason why a "poor" category does not exist for judge ratings, and that is that "poor" is not constructive feedback. Either you rate a judge well, or you tell them specifically why they didn't do well, or you don't rate them at all. That gives a fair overview of both how good judges are, and how they can improve. When somebody is unapologetically advocating breaking that system, they are actively trying to undermine the integrity of the site. They are making judge scores meaningless and breaking one of the key judging incentives. Without good judging, and judges with incentives to improve, you don't get good debating either. Why debate when the judges have no incentive to bother to try to do well? And when that happens, there's no point to there being a edeb8.

It follows that if a judge rating is being made in a justified fashion as opposed to an unhelpful one, there should be a readily available justification we can use. That doesn't mean it must be immediately explicated, but it should be there. Several times in the past I have explained my own judge rating on judgments where judges have asked for clarification on why their feedback was a certain way. I see that as a GOOD thing, because it shows a desire on the part of the judge to improve. Asking more questions on the topic is not harassment, it is simply a desperation to improve. And isn't that primarily what we're all about? Practicing debating, having fun doing it, and learning how to do better next time?

This matters a lot to me. I can't stress that enough. Maintaining the integrity of positive judging with a view to making all judges as constructive as possible has always been edeb8's goal. There's nothing more frustrating than to spend years thinking about and developing this system, only for it to be openly ignored by members to the detriment and division of the community. It's bringing down edeb8 and we have to stop it.
I'm the main developer for the site. If you have any problems, ideas, questions or concerns please send me a message.
Let's revive the forums!
Blackflag
By Blackflag | Jun 14 2015 3:57 PM
There is a reason certain features exist, and misuse of those features has always been grounds for moderation. This isn't necessarily an attack on your behavior, but on the position you have been advocating. I am a moderator first and an administrator second, whether you like it or not.

I'm suprised that you like it that way. It seems detrimental to your own goals to worry first about moderation and second about creating the best debate site possible.


Judgments exist so that debaters can improve. Disincentives to this exist in the form of judge scores, but you've made it absolutely clear you don't care about judge scores. Therefore you have no incentive to improve the quality of your judging, and this is played out in one-point decisions you have been making despite a proven capability to do higher-point decisions, an obvious abuse of the intention behind that system.

Judge scores are a good indicator of how good of a judge someone is, and how much their judgement should be valued. Nothing more or nothing less. It isn't like I don't care about my score. My ego just isn't large enough to the point where I obsess over it or get frustrated whenever it goes down.

In a similar fashion, judge ratings are supposed to make judges improve. There's a specific reason why a "poor" category does not exist for judge ratings, and that is that "poor" is not constructive feedback.
I know, I think that is an incredibly stupid oversight as well. A rating might help people improve at the most base levels, but in my opinion, which is certainly more correct than yours, judge ratings serve as a disincentive to make bad judgements. Being rated poor alone is enough of an incentive, constructive criticism is just an added bonus . Anyone who rates a judgement period is doing the site a favor.

Either you rate a judge well, or you tell them specifically why they didn't do well, or you don't rate them at all.

Dependent on me not giving a good rating, I am under no obligation to do either of those things, and most likely I will continue to not do either of those things.


This matters a lot to me. I can't stress that enough. Maintaining the integrity of positive judging with a view to making all judges as constructive as possible has always been edeb8's goal.

If you want to maintain the integrity of positive judging allow people to rate judgements without being liable to having to "explain" their judgement, which is really just code for "Get in a fight over why I rated a judgement one way over the other."

You don't realize it, but your ideas don't align with your goals.
Blackflag
By Blackflag | Jun 14 2015 4:05 PM
And how come you feel good is constructive but poor is not? That seems like a double edged sword.
admin
By admin | Jun 14 2015 5:11 PM
Blackflag: We are already ahead in terms of everything I as an administrator can do. What edeb8 lacks is effective moderation, as is clear given incidents such as this, and that's why I am a moderator first. That helps edeb8 be the best it can be more than anything else I could do right now.

Nobody is suggesting this is about big egos arguing to protect their scores. What this is about is judges genuinely looking for ways to improve their judgments on the basis of fair feedback. If anyone has an ego, it's the self-centered jerks who give non-constructive responses to open questions on feedback they have themselves provided. That's about as egoistic as it gets. In fact the only more egoistic thing I can think of right now is people who claim their opinions are certainly more right than everybody else's despite not providing a rationale. Functionally the two are much the same though.

I don't think you read what I wrote concerning why I'd never add a category for poor judgments. It's not an oversight, it's deliberate to force raters to provide feedback on their rationale. Edeb8 avoids disincentives to making judgments everywhere on the site (heck, that's why 1-point decisions exist), but without constructive feedback, a lower rating is inherently meaningless for a judge looking to improve. If judge ratings ever become a disincentive for poor judgments, I will scrap them. People don't set out to make poor judgments, people try hard and occasionally make mistakes. These judgments should be valued for their effort and, if we don't value them, then judges should be able to clearly understand what they need to do to become valued. So no, I don't agree that anyone who rates a judgment is doing the site a favor. If they're not helping the judge then the judge has no incentive to improve, and thus there is little incentive to judge, and thus the whole site breaks apart. Bad judgment ratings could easily be the death of edeb8 if they don't drive me insane first.

Your lack of a sense of obligation is ridiculous. Debate is all about fulfilling obligations, from the obligation to judge fairly to the obligation to have a high standard of conduct in a debate. Sometimes we fall short of these obligations, but we still must try to make them. If the lack of an obligation is preventing you from attempts at basic debate courtesy, then maybe that can be a problem to be remedied.

Asking for feedback is not code for fighting over whether a rating was legitimate, anymore than asking for a judgment is code for getting into a fight over who won a debate.

Your ideas are ruining the site.
I'm the main developer for the site. If you have any problems, ideas, questions or concerns please send me a message.
Let's revive the forums!
Blackflag
By Blackflag | Jun 14 2015 7:03 PM
Nobody is suggesting this is about big egos arguing to protect their scores. What this is about is judges genuinely looking for ways to improve their judgments on the basis of fair feedback. If anyone has an ego, it's the self-centered jerks who give non-constructive responses to open questions on feedback they have themselves provided. That's about as egoistic as it gets. In fact the only more egoistic thing I can think of right now is people who claim their opinions are certainly more right than everybody else's despite not providing a rationale. Functionally the two are much the same though.
Well this is a problem. You want to force people do do everything the way you want it to be done, while at the same time allowing features that people can use that go against the ways you prefer.
You shouldn't be forcing people to do things your way. You should be encouraging them. Big difference.

Anyways, you dropped my most essential point. There is merit in having a rating in itself, and if we both agree that ego is not the reason for your proposition, then you should actually be agreeing with me that the right to give an anonymous rating without further comment is for the best.

I don't think you read what I wrote concerning why I'd never add a category for poor judgments. It's not an oversight, it's deliberate to force raters to provide feedback on their rationale. Edeb8 avoids disincentives to making judgments everywhere on the site
Because feedback isn't more likely to be provided with a "poor" judgement? It is in fact an insane flaw and oversight that we can only rate judgements as good or better. If you can't admit that is an oversight than at least recognize the stupidity of it all.

Your lack of a sense of obligation is ridiculous. Debate is all about fulfilling obligations, from the obligation to judge fairly to the obligation to have a high standard of conduct in a debate.
Obligations in the truest sense of the word don't exist. I've said it a thousand times. The only thing I'm obligated to do is what I choose to do, and what I choose to do is limited by what you allow me to do, and as long as you allow me to cast an anonymous judgement without having to get in a fight with another use defending my decision, then I will continue to make anonymous judgements witout having to get in a fight with another use over my decision.

You are attacking a user with no administrative power over a design flaw that you can't own up too. This "mess" you keep claiming exists was created by you. Get real.

Asking for feedback is not code for fighting over whether a rating was legitimate, anymore than asking for a judgment is code for getting into a fight over who won a debate.
I give feedback all the time, especially on judge comments, but it isn't an obligation. When I genuinely feel someone wants my feedback and will value it, I will most likely give feedback. When I feel someone just wants to argue with me about how my opinion of their judgement is wrong, I will not give any feedback because it will lead to petty squabbling.

This is why you need to find a new moderator. Drama starts and you keep fueling it. Stop making nothing into something. Didn't you learn that administration and moderation never go well together?I love you as an administrator, but you need to give up on attempting to be a moderator #constructivecriticism
Blackflag
By Blackflag | Jun 15 2015 2:01 AM
Okay, thought about it for awhile, and it seems I may of been acting a little inappropriately in this thread in regards to how I argue and what I say.Look, I get where you're coming from @admin and it makes sense, but I strongly believe that forcing expectations on the community is not the way to go. Every feature you have in regards to judging makes sense. Changing it doesn't make since. I would be way more susceptible to ideas on how we can make a judge rating system that encourages constructive feedback and deals with my concerns on "feedback" turning into "fights."
admin
By admin | Jun 15 2015 12:59 PM
Blackflag: Alright, let's let cooler heads prevail. I do want to say, for the record, like it or not, there are a set of well-established expectations that everyone must follow: the site terms of use. Please follow those everybody. Edeb8 mostly survives on an honesty system.

Now let's return to the original purpose of the thread. Everybody start contributing ideas.
I'm the main developer for the site. If you have any problems, ideas, questions or concerns please send me a message.
Let's revive the forums!