Dunning–Kruger effect

Published on

Transcript:

A webcomic in four panels:

Panel 1: A very young knight, poorly equipped (oversized pants, topless, wooden sword and shield, rusty helmet), screams with overconfidence in front of a cave on a foggy night.

Knight: Master of Dragons! I challenge you to a duel, against ME, the greatest knight the kingdom has ever known!!! My knowledge of fencing is unrivaled!!!

Panel 2: The dragon walks calmly out of the cave, only the large nose is visible while his head is still in shadow. Not a fierce look, but a massive size. He is just tired of this kind of interruption. The knight continues:

Knight: You're terrified, aren't you?! well, you should be! I'm terrifying! Show Yourself!!! Coward!!!

Panel 3: The camera shows an oversized dragon, the knight is the size of a mosquito, attacking his toenails. The dragon just look at the situation with thinking. And an epic full moon illuminates the scene.

Knight: Hyah! Hyah! Partner of the dragon (off panel, from the cave): Honey, who is that?

Panel 4: A close-up of the dragon, turning his head towards the cave to answer his partner:

Master of Dragons: It's no one. As usual: an overconfident newbie, victim of overestimating his own abilities. A classic bias in human nature. Knight (off panel): Hyah! Hyah!


55 comments

link GuB 🕶️🎸🔌🎚️🔊💥   - Reply
gub


Je le reconnais ! C'est Jean-Kevin, le stagiaire de la hotline !

3 ★

link Riedler (2004) [E] [!]   - Reply
Riedler@donotsta.re

the dunning kruger effect has largely been disproven (or at least what people think it is) and people are generally ok at estimating their own abilities.

regardless, good comic 10/10

link Konrad Rudolph   - Reply
klmr@mastodon.social

@Riedler That’s not true. There was one recent paper that made a splash claiming this, but multiple studies over the years have corroborated the general existence of such an effect in various scenarios.

link Scien   - Reply
scien@social.linux.pizza

@klmr @Riedler There may be a different study that shows this effect, but the original paper was completely incorrect: it was autocorrelation.

As for a "general effect" - we have evidence that novices will misjudge their expertise more than experts, yes, but this goes *both* ways. The original paper simply claims they overestimated (or, at least, overestimated wildly more than they underestimated).

Also, it wasn't simply a *recent* paper that disproved the original - there's papers as early as 2016. That's 8 years ago.

link Scien   - Reply
scien@social.linux.pizza

@klmr @Riedler See: this thread I made here about it: social.linux.pizza/@scien/1126

If you know of any studies that indicate a similar conclusion as the original dunning-krueger effect study, I'd be quite interested.

link Riedler (2004) [E] [!]   - Reply
Riedler@donotsta.re

@scien @klmr honestly you just need to think about it a bit to disprove the original study. look at the graph and think about what it really means. yeah, nothing at all, really. With a bit of an asterisk, but it's definitely not what people generally think.

I'd love to do an explainer on this but I'm a bit scared of coming off as too, err… womansplainy? idk, I hope you know what I mean.

link Marc Tapages 🐰⏚🚲🎺   - Reply
marctapages

encore un à croire que c'est malin de dissoudre....

Je pense que je souffre d'un biais aussi :-\

link Gynux   - Reply
Gynux@the.goofs.space

Ah tiens, je ne connaissais pas cet effet ! Merci !
Pas sûr que je le retienne par contre...
Sur l'article wikipedia, l'exemple donné (à base de jus de citron) est assez saisissant et j'avoue que je n'aurais pas nécessairement décrit le protagoniste comme quelqu'un souffrant de "surconfiance" 🤣
Ceci dit, je suis sûr que ça m'est déjà arrivé dans une moindre mesure.

link Oregon Wine Woman   - Reply
NorCalWineLady@sfba.social


What came to mind? A classic Calvin and Hobbs tale. Still, good and funny at the same time.

link overbyte   - Reply
overbyte@gamepad.club

funny with beautiful art. #follow

link David Revoy Author, - Reply
davidrevoy

@overbyte Thank you!

link Marty Fouts   - Reply
MartyFouts@mastodon.online

DK is my favorite example of bad science that resonates so much that it’s impossible to convince people it’s wrong. The original paper had an error in the statistical analysis. It turns out that there is no correlation between intelligence and the ability to self evaluate it. Over confidence is just as likely among highly intelligent people as it is among everyone else.

link Scien   - Reply
scien@social.linux.pizza

@MartyFouts Interestingly, I believe there was a separate study that proved there was a phenomenon present about misjudging your capabilities - but it went both ways.

In short, the study concluded a lack of expertise (in a particular subject, not intelligence mind you) was correlated with both overestimating AND underestimating your capabilities.

Which makes a lot of sense. How do you know where you stand in terms of expertise without knowing of trends in the wider field?

link Mechanistisch Entmensch   - Reply
subpanel@mstdn.social

@scien @MartyFouts

On which task did the study measure performance?

link Scien   - Reply
scien@social.linux.pizza

@subpanel @MartyFouts

SCLI averages. SCLI stands for Science Literacy Concept Inventory - a 25 item test, which is outlined in this paper: journals.asm.org/doi/10.1128/j

I found the article I read on this: economicsfromthetopdown.com/20

In it, they cite a figure from:

(Original message has been truncated: read the complete original message here.) 🖼️ d44818e9f318bcff.png 

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@scien @subpanel And there’s an error in the interpretation of the analysis of the data. It’s not exactly the same error as in the original DK paper but it’s from the same class of errors.

They are showing an autocorrelation between degree of expertise and range of expertise that matches both the actual and estimated self evaluation.

link Scien   - Reply
scien@social.linux.pizza

@MartyFouts @subpanel

Pardon? I'm not sure I understand. Are you saying this study I linked connects the expertise factors and self-evaluation and thusly results in auto-correlation somehow?

If so, could you elaborate? Afaik the variables are quite separate. Here's another graph demonstrating the effect:

🖼️ 56c33efab2a6b852.png 

link Scien   - Reply
scien@social.linux.pizza

@MartyFouts @subpanel

Perhaps the effect is best showcased here. The vertical axis indicates the recorded *error* in estimation of abilities compared with the actual results.

The black dots are the individual estimates (the green zones are the means afaik). Thusly, you can see how the professors group has much more reduced spread, than, for instance, the freshman group.

link Scien   - Reply
scien@social.linux.pizza

@MartyFouts @subpanel

Now tbf the study still has its limitations (for instance you can see how the professor count is a pretty low 69), but I'm not sure autocorrelation is one of them... especially considering the authors went at great length to prove this of the original dunning-krueger in the very same paper.

I'd be very interested in learning why, if this was actually the case.

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@scien @subpanel I think that either interpretation is reasonable and without the raw data we can only rely on our intuition. But I think that it is an experiment design problem.

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@scien @subpanel The range of skills that they use to describe someone as “expert” is small. The range of skills that they use to describe someone as non expert is large. If accuracy of self assessment was dependent on skill level then the least square differences would be much smaller for experts then they are and the range for non experts larger. That the ranges of actual and self assessment match indicates autocorrelation. /1

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@scien @subpanel You need to do correlation calculations on the raw data to show that but intuitively it is similar to the error in the original study.

link Scien   - Reply
scien@social.linux.pizza

@MartyFouts @subpanel

Is it possible that this has very little effect on the end results? In the paper the authors quite literally input randomized data to show that it doesn't fail to the same fault.

link Scien   - Reply
scien@social.linux.pizza

@MartyFouts @subpanel

🖼️ ace2084b128fbe72.png 

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@scien @subpanel Possible, yes. I just think that emphasizing the size of the self evaluation set versus the measure set is misleading. I suppose I could write the authors and ask for the raw data but I don’t have the resources now that I’m retired.

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@scien @subpanel In a related note: one thing I know from teaching is that experts are terrible at predicting how hard problems will be for learners.

link Scien   - Reply
scien@social.linux.pizza

@MartyFouts @subpanel :P That reminds me of an XKCD panel: xkcd.com/2501/

Btw - thanks for the insight!

It really is quite hard to be sure you're interpreting a paper right, let alone that the paper itself doesn't have some intrinsic issue itself, without the relevant knowledge.

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@scien @subpanel There’s always an XKCD. Thanks for reminding me of that one.

link Mechanistisch Entmensch   - Reply
subpanel@mstdn.social

@MartyFouts @scien

Please tell M. E. the test scores used in the study weren't about estimating "the average person's familiarity" with anyone's field.

link m_on_stair   - Reply
m@miruku.cafe

@MartyFouts@mastodon.online @davidrevoy@framapiaf.org
oh? i was under the impression that it corretly claimed that people who are not average (either significantly worse or better than average) tend to guess their abilites lean more towards average.

what you said is interesting and i wouldnt be surprised if its true, do you have a source?

link Jamie Richardson   - Reply
writingshrink@mastodon.social

@m @MartyFouts Marty may have other sources but this one is a good read: economicsfromthetopdown.com/20

link m_on_stair   - Reply
m@miruku.cafe

@writingshrink@mastodon.social @MartyFouts@mastodon.online @davidrevoy@framapiaf.org
this is very nicely put!
turns out what I thought dunning kruger was was a little bit more sensical than what they claimed, but was still wrong!
cool stuff!

link Kelson   - Reply
kelson@notes.kvibber.com

@MartyFouts I was always under the impression that the effect was describing a correlation between competence at a specific skill and the ability to evaluate it, not overall intelligence.

link Marty Fouts   - Reply
MartyFouts@mastodon.online

@kelson It was, but the popular telling has simplified the claim. The actual claim was wrong for the reason I gave: what they thought was a correlation was an error in their analysis.

link OldGeek   - Reply
oldgeek@masto.yttrx.com

@MartyFouts

Yup. I believe SV is full of 'em

link Angry Sun   - Reply
sun@shitposter.world

he's a kid, the dragon should encourage him to get better!

link MHunt   - Reply
mhunt@socel.net

you should read the "Dragon Knights"

link David Revoy Author, - Reply
davidrevoy

@mhunt Thank you for the recommendation. This one? en.wikipedia.org/wiki/Dragon_K

link MHunt   - Reply
mhunt@socel.net

no, this one:

amazon.co.uk/Chronicles-Dragon

It was published by Delcourt but I went to their website and couldn't find it.

link   - Reply
derle

@mhunt La geste des chevaliers dragons ?

link MHunt   - Reply
mhunt@socel.net

@derle yes. I only know the English title. I think it doesn't has new volumes.

link   - Reply
derle

@mhunt It has 32 volumes in French ^^ (the editor is Soleil, not Delcourt, even if Delcourt bought Soleil their genre are a bit different)

link David Revoy Author, - Reply
davidrevoy

@derle @mhunt Wow, that's a large saga. i'll try to keep the title in mind when I visit my book store.

link   - Reply
derle

It's very Soleil through, you have been warned ^^ @mhunt

link m_on_stair   - Reply
m@miruku.cafe

@davidrevoy@framapiaf.org
the comic is great but this isnt really dunning kruger

link Fred   - Reply
fredds@mamot.fr

si je peux me permettre : le dessin ne ressemble pas trop a Macron, pourtant je suis sur que c'est lui qui est décrit, non? ^^

link Vincent Cantin   - Reply
greenCoder@functional.cafe

"This is but a scratch" ^_^

🖼️ 90d1275a367f83eb.png 

link xuxxux   - Reply
xuxxux@social.tchncs.de

@greenCoder aaaah the classics.

King Ohhh....very nice...I didn't vote for you

- You don't vote for the king.

How'd you get to be king anyway?

link xuxxux   - Reply
xuxxux@social.tchncs.de

@greenCoder

- The lady of the lake, her arm clad in the finest samite, thrust forth Excalibur from the bosom of the water thereby declaring by divine right that, I, Arthur should be your king.

Listen, women laying around in ponds distributing swords is no
basis for a system of government. Supreme executive power is
derived by a mandate from the masses not by some farsical aquatic
ceremony.

link Gen X-Wing   - Reply
breadbin@bitbang.social

The wooden sword gets me:)

link Frischling   - Reply
Frischling@wehavecookies.social

@anatoliyl no, those knights only exist, because they smell bad and don't taste.

link Jean-Marc Courtiade   - Reply
jmcourtiade

Nice one. Also known as the Dunning–Kruger effect (this is terrible with under-aged bosses or managers...)

link Chakat Firepaw   - Reply
chakatfirepaw@universeodon.com


(A week later.)

"Again? Let me guess, you are another great knight?"
"Me? No no no, I'm just a high school student from another universe who has...."
"HONEY! Pack the hoard and get out of here!"

link elCelio 🇪🇺 🇺🇦   - Reply
elCelio@mastodon.uno

an annoying mosquito


Post a reply

The comments are synchronised every 12h with the replies to this post on Mastodon:


How to use this? (click here to unfold)
Open a new Mastodon account on the server of your choice. Then, Copy/Paste the adress above in your Mastodon 'Search' field. The post will appear and you'll be able to fully interact with it. You'll have full control of your posts: edit, remove, etc. After that, your message will appear here.

Just please note that it may take up to 12 hours for your changes to be reflected here.