Pages in topic:   [1 2 3] >
Feature suggestion: Individual KudoZ efficiency metric
Thread poster: Boris Shapiro
Boris Shapiro
Boris Shapiro
Russian Federation
Local time: 21:32
English to Russian
+ ...
Apr 18, 2019

Hi!

I wonder if it is planned to introduce some kind of efficiency metric regarding the way individual members answer terminology questions. I mean, instead of showing the total number of point, show the points to answers given ratio. 'Cause clearly some colleagues are just spamming the system with their answers on the off-chance that they actually hit it. And while that in and of itself is just a nuisance, nothing more, with the way total KudoZ points are used to rank people here t
... See more
Hi!

I wonder if it is planned to introduce some kind of efficiency metric regarding the way individual members answer terminology questions. I mean, instead of showing the total number of point, show the points to answers given ratio. 'Cause clearly some colleagues are just spamming the system with their answers on the off-chance that they actually hit it. And while that in and of itself is just a nuisance, nothing more, with the way total KudoZ points are used to rank people here this turns out to be very much misleading!
Collapse


Grace Anderson
 
Jared Tabor
Jared Tabor
Local time: 15:32
SITE STAFF
Answer acceptance rate? Apr 18, 2019

HI Boris,

If you go to a person's KudoZ activity in their profile, there is a tab called "Stats" where you can see, among other things, their answer "acceptance rate", which shows the percentage of answers selected as most helpful against the number of questions answered. Is that what you mean?

Jared


 
Boris Shapiro
Boris Shapiro
Russian Federation
Local time: 21:32
English to Russian
+ ...
TOPIC STARTER
Can you post a screenshot? Apr 18, 2019

Hi Jared,

I'm confused. Whenever I click on someone else's KudoZ activity, I only see Point totals, Asked, Opened, and Answered tabs. No 'Stats' tab and no acceptance rate.

And even if there is such a value, aren't people still rated by their point totals?


 
Thomas T. Frost
Thomas T. Frost  Identity Verified
Portugal
Local time: 18:32
Danish to English
+ ...
Stats only in my own profile Apr 18, 2019

The Stats information is good, but I'm only shown this tab in my own profile, not others'. Is that how it should work?

Hedwig Spitzer (X)
Maria Teresa Borges de Almeida
 
Jared Tabor
Jared Tabor
Local time: 15:32
SITE STAFF
My mistake, on two counts Apr 18, 2019

Sorry about that, I thought the Stats were shown, but 1) they are only available to paying members and 2) they are only shown to the profile owner, not publicly.

What do others think about Boris' suggestion?



Added: ranking is done by point totals, yes. There could also potentially be an option to rank by point total and acceptance rate, for example.

Jared


 
Thomas T. Frost
Thomas T. Frost  Identity Verified
Portugal
Local time: 18:32
Danish to English
+ ...
What I think Apr 18, 2019

Jared Tabor wrote:

What do others think about Boris' suggestion?

Added: ranking is done by point totals, yes. There could also potentially be an option to rank by point total and acceptance rate, for example.


I agree with the overall idea, but a simple ratio would be misleading. Let's say I just answer one question right as my only Kudoz activity ever, and I'd have a 100% success rate.

It's a complicated matter, which requires careful analysis.

There should probably be some 'decay' of old data, so that a question answered 20 years ago doesn't count as much as a question answered recently (and similar for questions asked). For each year, each parameter could lose x% of its weight.

Then there is Boris' suggested answered-to-asked ratio, which is a good factor.

There is also the success rate out of answered questions.

And something else that sometimes bothers me about Kudoz: some members quickly get unpleasant if you don't agree with them. They may take a 'disagree' vote as a personal insult and attack the voter. There is too much uncivilised behaviour. I would suggest that if a moderator has to step in against a member for such a reason, it would result in a Kudoz-point penalty that can be felt, so as to deter that type of behaviour. Of course, abusive ‘disagree’ votes could also be penalised, i.e. if someone votes a suggestion down without being able to provide any linguistic justification.


Yoana Ivanova
Hedwig Spitzer (X)
Andy Watkinson
Elizabeth Tamblin
mk_lab
Maria Teresa Borges de Almeida
 
Yoana Ivanova
Yoana Ivanova  Identity Verified
Estonia
Local time: 20:32
English to Bulgarian
+ ...
Before that Apr 18, 2019

Such a feature could indeed be useful. However, I think that first some kind of better system should be implemented to ensure that Kudoz questions don't stay open indefinitely without an answer ever being chosen.

I know that the asker gets reminded via email and then questions can be closed automatically based on agrees, but when neither of those work out, they can remain open forever. That in turn automatically brings down our answer acceptance rate.


Thomas T. Frost
Platon Danilov
Michele Fauble
 
Katalin Horváth McClure
Katalin Horváth McClure  Identity Verified
United States
Local time: 14:32
Member (2002)
English to Hungarian
+ ...
@Jared - take a stroll down Memory Lane. :-) Apr 18, 2019

This topic has been discussed numerous times, and there were multiple proposals on how to incorporate the acceptance ratio into the directory.
Here is one link, but you can do a search for +acceptance +rate and you will get more.
https://www.proz.com/forum/kudoz/119297-hit_rate_or_points_to_closed_answers_ratio_pta.html#980033


Sheila Wilson
Yolanda Broad
Cilian O'Tuama
Andy Watkinson
Erik Freitag
mk_lab
Mirko Mainardi
 
Thomas T. Frost
Thomas T. Frost  Identity Verified
Portugal
Local time: 18:32
Danish to English
+ ...
And … Apr 18, 2019

The detailed stats should probably remain private, but a compound score based on weighted parameters would probably be more useful than just the total score.

In order for each member to evaluate their own relative score, it would be helpful to include the global average for ‘acceptance rate’, ‘average agrees per question’ and ‘average disagrees per question’ in each category in the private stats.


 
Manuel Bas y Mansilla
Manuel Bas y Mansilla  Identity Verified
Netherlands
English to Spanish
Peer review Apr 19, 2019

As someone mentioned, it's been discussed before, and one suggestion was to have it based not on chosen answers (it'd be misleading and might discourage participation) but on peer agree/disagree.
No system is perfect, and certain areas get less participation from colleagues, but there should be some 'penalty metric' for people who constantly get downvoted by various members (as to avoid personal issues).


mk_lab
Platon Danilov
 
Andy Watkinson
Andy Watkinson
Spain
Local time: 19:32
Member
Catalan to English
+ ...
Decay? Yikes! Apr 19, 2019

Thomas T. Frost wrote:


There should probably be some 'decay' of old data, so that a question answered 20 years ago doesn't count as much as a question answered recently (and similar for questions asked). For each year, each parameter could lose x% of its weight.



I entirely understand your point about proposing just one answer, getting it right and thus being hailed as a 100% genius. There are so many factors involving language combinations, ratio asked/answered, ratio answers right/rejected, accumulated agrees/disagrees, etc..that my CSE maths-level brain hurts.

But what I don't understand is this idea of attaching more importance to recent answers than those from xx years ago.

I joined ProZ about 18 years ago, discovered KudoZ, thought "Eh up, lad! This looks like fun" and immediately started answering anything that moved. I acquired a couple of thousand points fairly quickly, continued for a bit and over recent years probably average a couple of answers a month, if that.

So, my recent contributions might not add up to much but I don't see my sixty-something rate of chosen answers in Catalan or fifty odd in Law* being any less valid or useful now than they were when asked. Someone might well have used one of them yesterday for all I know, the same way I obtain valuable information from answers posted by former colleagues who even left the site years ago.

i.e., their knowledge, experience, reliability are no less valid now than then.
In fact, all three are probably greater now than then, given the time elapsed. Afer all, you don't say "Hendrix hasn't played anything for 50 years. Francis Rossi is now a better guitarist."

And I'm decaying quite nicely all on my sweet own, thank you.
I don't need my reputation following suit.

------o------

* Thanks to whoever pointed out that these Stats existed - I had no idea until now.

No te acostarás sin saber una cosa más.

[Edited at 2019-04-19 04:00 GMT]

[Edited at 2019-04-19 14:27 GMT]


Kay Denney
MoSpa
Tina Vonhof (X)
Michele Fauble
Sandra & Kenneth Grossman
Recep Kurt
 
Lincoln Hui
Lincoln Hui  Identity Verified
Hong Kong
Local time: 02:32
Member
Chinese to English
+ ...
Throw out the baby, keep the bathwater! Apr 19, 2019

Great idea, it will discourage people from participating and helping others, the primary goal of KudoZ, in order to maintain the integrity of the points system, a secondary goal.

[Edited at 2019-04-19 04:21 GMT]


mk_lab
Michele Fauble
Anaïs G.
 
Sandra & Kenneth Grossman
Sandra & Kenneth Grossman  Identity Verified
Israel
Local time: 20:32
French to English
+ ...
No. Apr 19, 2019

Jared Tabor wrote:

What do others think about Boris' suggestion?

There could also potentially be an option to rank by point total and acceptance rate, for example.

Jared


If it ain't broke don't fix it.

Each language and direction has its own participants; some in the thousands, and some hardly any. In popular language pairs, there is lively participation and competition for points, in others, it's not even worth posting questions...

This means that the chance of acceptance in popular languages is much lower, for the same number of answers. Even for a correct answer, somebody rushing in to give an earlier answer may be superseded by someone else chiming in with a more appropriate synonym.

I enjoy the game of Kudoz and the confrontation of ideas it generates. Leave it as it is. It's fun.

I also think that posting the ratio of accepted/given answers would be a form of public shaming and should not be acceptable, even if some individuals give irrelevant answers.

Sandra

[Edited at 2019-04-19 07:34 GMT]


Elizabeth Tamblin
mk_lab
ahartje
Maria Teresa Borges de Almeida
Michele Fauble
Baran Keki
Anaïs G.
 
Daniel Frisano
Daniel Frisano  Identity Verified
Italy
Local time: 19:32
Member (2008)
English to Italian
+ ...
KudoZ doesn't make sense anymore Apr 19, 2019

When KudoZ started, answers were mostly based on actual skills: knowledge, experience, deciphering obscure sentences, intuition.

Now it's just a race to whoever gets the most Google hits.

Agrees are awarded and exchanged (and thanked, as a personal favour) between buddies to boost each other's egos.

To make matters worse, the staff won't do a thing against multiple answers by the same user and other aggressive practices.

It has become a joke.
... See more
When KudoZ started, answers were mostly based on actual skills: knowledge, experience, deciphering obscure sentences, intuition.

Now it's just a race to whoever gets the most Google hits.

Agrees are awarded and exchanged (and thanked, as a personal favour) between buddies to boost each other's egos.

To make matters worse, the staff won't do a thing against multiple answers by the same user and other aggressive practices.

It has become a joke. But hey, welcome to 2019, when being liked is everything that matters.

(By the way, no "likes" on this post please. It is all SO childish).
Collapse


writeaway
Platon Danilov
Beatriz Ramírez de Haro
 
Elizabeth Tamblin
Elizabeth Tamblin  Identity Verified
United Kingdom
Local time: 18:32
French to English
Bad idea Apr 19, 2019

Jared Tabor wrote:

Sorry about that, I thought the Stats were shown, but 1) they are only available to paying members and 2) they are only shown to the profile owner, not publicly.

What do others think about Boris' suggestion?



Added: ranking is done by point totals, yes. There could also potentially be an option to rank by point total and acceptance rate, for example.

Jared


When I use the Term Search feature, I like to see a range of suggestions. I don't always choose the term that was awarded points, because one of the other suggested terms might be more suitable in the context.

Just because a term is not "the winner" does not mean it is an invalid or a bad suggestion. Often there are several valid suggestions, and it would be unfair to penalise people who are being very helpful to their colleagues be devaluing their contributions.


Sandra & Kenneth Grossman
Platon Danilov
Maria Teresa Borges de Almeida
Grace Anderson
Thomas T. Frost
mk_lab
Michele Fauble
 
Pages in topic:   [1 2 3] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Feature suggestion: Individual KudoZ efficiency metric






TM-Town
Manage your TMs and Terms ... and boost your translation business

Are you ready for something fresh in the industry? TM-Town is a unique new site for you -- the freelance translator -- to store, manage and share translation memories (TMs) and glossaries...and potentially meet new clients on the basis of your prior work.

More info »
Anycount & Translation Office 3000
Translation Office 3000

Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.

More info »