The Zipf Mystery

15. sep. 2015
19 050 856 Ogledi

The of and to. A in is I. That it, for you, was with on. As have ... but be they.
RELATED LINKS AND SOURCES BELOW!
twitter.com/tweetsauce
instagram.com/electricpants
WordCount.org www.wordcount.org/
How many days have you been alive? www.beatcanvas.com/daysalive.asp
random letter generator: www.dave-reed.com/Nifty/randSeq.html
Dictionary of Obscure Sorrows: slworlds.info
Word frequency resources:
[lemmatized] en.wikipedia.org/wiki/Most_common_words_in_English
www.uow.edu.au/~dlee/corpora.htm
www.wordfrequency.info
www.anc.org/data/anc-second-release/frequency-data/
www.titania.bham.ac.uk/docs/
www.kilgarriff.co.uk/bnc-readme.html#raw
en.wiktionary.org/wiki/Wiktionary:Frequency_lists
ucrel.lancs.ac.uk/bncfreq/
[PDF] www.wordfrequency.info/files/entries.pdf
[combined Wikipedia and Gutenberg] www.monlp.com/2012/04/16/calculating-word-and-n-gram-statistics-from-a-wikipedia-corpora/
corpus.byu.edu/coca/files/100k_samples.txt
corpus.byu.edu/
corpus.leeds.ac.uk/list.html
books.google.co.uk/books?id=ja1_AAAAQBAJ&dq=word+frequency+coca&lr=
www.ling.helsinki.fi/kit/2009s/clt231/NLTK/book/ch01-LanguageProcessingAndPython.html
Great Zipf's law papers:
colala.bcs.rochester.edu/papers/piantadosi2014zipfs.pdf
www.ling.upenn.edu/~ycharles/sign708.pdf
arxiv.org/pdf/cond-mat/0412004.pdf
www-personal.umich.edu/~mejn/courses/2006/cmplxsys899/powerlaws.pdf
Zipf’s law articles and discussions:
www.theatlantic.com/magazine/archive/2002/04/seeing-around-corners/302471/
io9.com/the-mysterious-law-that-governs-the-size-of-your-city-1479244159?.Ej9760cOTJCPS_Bq4mjoww.0
plus.maths.org/content/os/latestnews/may-aug08/food/index
judson.blogs.nytimes.com/2009/05/19/math-and-the-city/?em
plus.maths.org/content/mystery-zipf?src=aop
www.datasciencecentral.com/profiles/blogs/why-zipf-s-law-explains-so-many-big-data-and-physics-phenomenons
en.wikipedia.org/wiki/Zipf%27s_law
books.google.co.uk/books?id=f8GrzlnMSm8C&pg=PA62&redir_esc=y#v=onepage&q&f=false
arxiv.org/pdf/0802.4393v1.pdf
www.pnas.org/content/108/9/3526.full
lewisdartnell.com/language_page.htm
wugology.com/zipfs-law/
other Zipf’s law PDFs
ftp.iza.org/dp3928.pdf
arxiv.org/pdf/1402.2965.pdf
arxiv.org/pdf/1104.3199.pdf
www.lel.ed.ac.uk/~jim/zipfjrh.pdf
www.ncbi.nlm.nih.gov/pmc/articles/PMC2834740/#pone.0009411-Mandelbrot1
polymer.bu.edu/hes/articles/pgs02a.pdf
in untranslated language: arxiv.org/pdf/0808.2904.pdf
pages.stern.nyu.edu/~xgabaix/papers/zipf.pdf
www.hpl.hp.com/research/idl/papers/ranking/ranking.html
statweb.stanford.edu/~owen/courses/306a/ZipfAndGutenberg.pdf
arxiv.org/pdf/1310.0448v3.pdf
www.kornai.com/Papers/glotto5.pdf
Zipf’s law slides:
www.slideshare.net/guest9fc47a/nlp-new-words
Pareto Principle and related ‘laws’:
www.squawkpoint.com/2013/03/pareto-principle/
billyshall.com/blog/post/paretos-principle
en.wikipedia.org/wiki/Pareto_principle
Random typing and Zipf:
www.longtail.com/the_long_tail/2006/09/is_zipfs_law_ju.html
health 80/20: archive.ahrq.gov/research/findings/factsheets/costs/expriach/expriach1.html
Principle of least effort:
en.wikipedia.org/wiki/Principle_of_least_effort
en.wikipedia.org/wiki/Satisficing
www.pnas.org/content/100/3/788.full.pdf [PDF]
csiss.org/classics/content/99
self organized criticality:
journal.frontiersin.org/article/10.3389/fnsys.2014.00166/full
Hapax Legomenon:
campus.albion.edu/english/2011/02/15/hapax-legomenon/
www.dailywritingtips.com/is-that-a-hapax-legomenon/
en.wikipedia.org/wiki/Hapax_legomenon
[PDF] www.aclweb.org/anthology/J10-4003
www.wired.com/2012/01/hapax-legomena-and-zipfs-law/
oed.hertford.ox.ac.uk/main/content/view/402/450/index.html#_ftn1
oed.hertford.ox.ac.uk/main/content/view/36/166/index.html
Learning curve: en.wikipedia.org/wiki/Learning_curve
Forgetting curve:
www.trainingindustry.com/wiki/entries/forgetting-curve.aspx
en.wikipedia.org/wiki/Forgetting_curve
Experience curve effects: en.wikipedia.org/wiki/Experience_curve_effects
Forgetting
and zipf's law: act-r.psy.cmu.edu/wordpress/wp-content/uploads/2012/12/37JRA_LS_PS_1991.pdf
public.psych.iastate.edu/shacarp/Wixted_Carpenter_2007.pdf
marshalljonesjr.com/youll-remember-less-than-001-of-your-life/
en.wikipedia.org/wiki/Forgetting
www.reddit.com/r/Showerthoughts/comments/3gu9qk/it_only_takes_three_generations_for_you_to_be/
music from:
slworlds.info
www.audionetwork.com

Komentarjev
  • What mumbo jumbo is doin here

    Mantvydas KazakeviciusMantvydas KazakeviciusPred 9 minutami
  • Everything has a 80% ≥ 20% it's kinda weird...

    SaedrisSaedrisPred uro
  • me when the. when the

    KillerQueenKillerQueenPred uro
  • the

    TimeSFTimeSFPred uro
  • I'll simplify this video for you. 🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️🤷‍♂️

    Shorey BanksShorey BanksPred 3 urami
  • I've been being recommended this for years and I'm finally bored enough to watch it

    BjornHubBjornHubPred 3 urami
  • Just convinced me we are definitely in a simulation

    17Canine1717Canine17Pred 5 urami
  • It was fun to see that you said “the” twice as often as “of”.

    John JohnsonJohn JohnsonPred 5 urami
  • sauce

    CodyCodyPred 5 urami
  • I think it goes back to the brain itself and the physics behind how consciousness is encoded...we have no choices but to ultimately produce this sequence. Maybe. Admittedly, first I heard of this.

    Vikram GuptaVikram GuptaPred 5 urami
  • They got lazy with the simulation...

    Jonah 2007Jonah 2007Pred 7 urami
  • One of the best.

    LordVoldemort's sidechickLordVoldemort's sidechickPred 9 urami
  • The soggy t-shirt reilly drain because description spontaneously travel on a aquatic study. acrid, dull tax

    Jacob CassarJacob CassarPred 12 urami
  • WTF

    garfieldgarfieldPred 15 urami
  • the example in 18:30 probably shows more how's Zipf's law is "our brain".. but actually there are people who actually can remember each day of their lives. However, now I realize that the number of people that remember only few days of their lives and the number of people that remember most days of their lives, also obeys Zipf's law. Damn, it's undeniable!!! xD

    KFoxInKFoxInPred 15 urami
  • Hey Michael, long time follewer here! I love the watch you are wearing in this video, can you tell me which brand and model it is? You’ll probably won’t see this comment but I’ll try my chances. Thanks in advance!

    Turan MüsevitoğluTuran MüsevitoğluPred 20 urami
  • Zipfian = the accumulation of advantage in language? Interesting.

    BeyondfubarBeyondfubarPred 23 urami
  • 80% of the views come from 20% of subscribers...20% of people in your life are the real ones

    Dumbass ProductionsDumbass ProductionsPred dnevom
  • When all of the comments use the word "the".

    United Armed Forces ConglomerateUnited Armed Forces ConglomeratePred dnevom
  • I don't get calls

    Billu #mnbvcBillu #mnbvcPred dnevom
  • Tinder's law: 1% of males get 100% of women

    Piotr BPiotr BPred dnevom
  • I don't know who you are or where you came from but I found this channel 2 days ago and now I'm crossed eyed.

    BroharnBroharnPred dnevom
  • 18:48 Whoa whoa whoa whoa! Michael's birthday is on January 23rd? How did I just happen to receive my first Curiosity Box, and feel inclined to go back and re-watch a bunch of Vsauce videos on Michael's birthday?

    MoolaMoolaPred dnevom
  • Me: "I don't remember most of what I read in books". Also me, 20 years after reading A Wrinkle In Time: "C is the square of the velocity of light in centimeters per second."

    Kazzy and DecchanKazzy and DecchanPred dnevom
  • Mind blown.

    Jane AJane APred dnevom
  • I’m guessing 20% of the SLworlds channels have 80% of the total views on SLworlds?

    David WilliamsDavid WilliamsPred dnevom
  • Great RW Emerson quote.

    Len PalmeriLen PalmeriPred dnevom
  • kdasj fkhdsi; kdnwejf; saijf sdmfjsda; fjkl huijosd hfjmsbfjksl i zfkjmje snj fhgeuj fusiod fkjmsd hjfsd fjksdl; fjks bfnjs bfvgudsxfv usdfzjk dns mklf hesdujvfifsd zkjfl;dhsjk mZfbjsl fujsd fjsdm hzfjdlhv iujsd fjmsd ;b jffls duvjsd jfghsdjv l sjkfhsd j FBvhdjsl VHfhdsLK F This can't be Zipfy. After all, I have only 2 thumbs to hit the space bar with, and 8 fingers to hit other letters with. So I am actually more likely to have short words, but VERY short words are rather unlikely due to the time it would take to use a thumb twice rather than use any of the other four fingers instead of the second thumb stroke. Also 16:15 I'm not sure that note is legal. If we are talking about words, I would expect to talk about _each unique word manifestation_ . Otherwise you are just defining the rank of certain types of words, like "article adjectives are the most common, prepositions are the second most, conjuntions are the third most," rather than the words themselves. That is just an illustration of how human communication trends are, which is obviously independent of language itself, because humans all conceptualize and use language the same way. "Say the most content, with a limited density, a maximum simplicity, and the shortest amount of time/syllables possible". And that's just Dijkstra's algorithm, with a few modifications I'm sure. Quizzaciously is just a hard word to say, and it's long. So, of course we wouldn't use it. If anything, Zipf-ian systems are just systems of priority, like Dijkstra's algorithm. You pick the favorable quality, and you ignore the rest because you don't have the processing power to keep it all. You forget the mundane, but all things deemed relevant or important are remembered. And occasionally, random facts are also randomly selected to be "included". If I'm a hiring manager, and I want to hire someone with X, Y, and Z qualities, then whatever resumes I see with X, Y, and Z will get the most of my time, and those without one or more of those desirable traits will receive very little, if any, of my time. It's just a prioritization system. A naturally occurring prioritization system.

    Hayato TenryuuHayato TenryuuPred 2 dnevi
  • 80% of the battery usage is by 20% of the processes.

    Andrew QiAndrew QiPred 2 dnevi
  • sauce

    Ikenna OkonkwoIkenna OkonkwoPred 2 dnevi
  • Was "the" a pun?

    Dead Guy Dude ManDead Guy Dude ManPred 2 dnevi
  • I wonder what a book that was intentionally written to not follow this would be like?

    Jacob RogersJacob RogersPred 2 dnevi
  • Wordcount uses Flash. And they haven't updated the site.

    Jonah FalconJonah FalconPred 2 dnevi
  • Isn't this mathematically normal? the total odds/frequency of anything will be 1 and when ranked by order in a graph will appear the same way? otherwise the total odds will either exceed or never reach 1, which can't happen.

    Aladen ZannerniAladen ZannerniPred 2 dnevi
  • The most thought word must be "F" word though,🤔

    Prakhar SharmaPrakhar SharmaPred 2 dnevi
  • I think the most used Hungarian word is "B@zdmeg" (i needed to censor it). It's a swear word. And it's beautiful.

    Bodor JánosBodor JánosPred 3 dnevi
  • "Quizzaciously" is more popular now. There is an article in Urban Dictionary.

    Dron008Dron008Pred 3 dnevi
  • duh!?

    Somenath GaraiSomenath GaraiPred 3 dnevi
  • wait...so does this mean 20% of all youtubers are actually making videos and are popular while 80% are only watchers ? does this means 20% out of those who make videos in youtube are popular and 80% are barely known ? does this mean 20% out of all of the comments in a video gets popular while 80% just fly by and barely get over 5 likes ? ...does this mean 20% out of everyone in this world...get rich while 80% just die poor does this mean 20% of people in this world die alone while 80% the opposite ? 20 80 rule is...scary and weird tbh it's just way too weird to think about it...this thing really did caught my attention

    kiro cazamkiro cazamPred 3 dnevi
  • Any German here? I Like the Title😂

    aabicaabicPred 3 dnevi
  • Ancient language that hasn't been translated yet-

    Anex AntonyAnex AntonyPred 3 dnevi
  • Wth is this wizardry! I was just thinking this morning about this vsauce video I’d seen years ago but I couldn’t remember what it was called! I only remembered the final quote from Emerson. And SLworlds just suggest this to me today. 🤯

    5MinPhil5MinPhilPred 3 dnevi
  • Vsauce: sauce is the 5,555 th mosts common english word Also Vsauce every video: 0:00

    Doby AdobyDoby AdobyPred 4 dnevi
  • The 45° angle line we get when graphing the frequency of words is not just for humans. The different sounds made by Dolfin show this 45° angle pattern as well.

    UndefindUndefindPred 4 dnevi
  • 80% of my brain is fried by the first 20% of this video

    Dish Soap GamingDish Soap GamingPred 4 dnevi
  • Idk if you're reading comments but what do you think, would a auto suggest feature on keyboard follow zipfs law. If we humans follow his law everywhere what's the possibility of computer with auto suggest for words doing the same? (i mean there is chance it would because we created it)

    N!ckyN!ckyPred 4 dnevi
  • So, flipping someone the bird while driving is responsible for about 95% of the road rage that occurs. Makes sense...

    Zach SchmiedZach SchmiedPred 4 dnevi
  • Most things go Top to bottom, you don't start a cake with a "hint of cinnamon " or " a cherry" even someone doing there accounts would start with big number first.

    Robcop WesttonRobcop WesttonPred 4 dnevi
  • would have been funny if he didnt say the for the entire video

    Pigbelly19 ‘Pigbelly19 ‘Pred 4 dnevi
  • mfw i suddenly realise the background music at 12:45 is actually bach double but jaunty piano mode

    Jacob CostenJacob CostenPred 4 dnevi
  • Yeah its simple.

    ِِPred 4 dnevi
  • Where did I put my paper clips?

    Toms MarkovsToms MarkovsPred 4 dnevi
  • Answer next tk questions

    ِِPred 4 dnevi
  • A similar law is Benford's Law.

    Vineet SharmaVineet SharmaPred 4 dnevi
  • me and the bois going on to wikapedia and making scauce the 69 most used word

    Henry AndersonHenry AndersonPred 4 dnevi
  • Reading up on superspreading events and epidemiology, the 80/20 rule was also observed -- 80% of disease transmission is caused by 20% of infected cases

    TheMockingJayTheMockingJayPred 4 dnevi
  • there is no way "school" is uses more than "no"

    Cameron GoodallCameron GoodallPred 5 dnevi
  • The Bible was written by man bc it follows ziffs law

    blocboy jbblocboy jbPred 5 dnevi
  • we bouta get quizzacious

    Matt GarreltsMatt GarreltsPred 5 dnevi
  • This is Benford's law

    Andrej AjnikAndrej AjnikPred 5 dnevi
  • The absent reason putatively guide because transaction regretfully milk besides a overrated territory. aboard, tranquil skirt

    Andrew ZeitlerAndrew ZeitlerPred 5 dnevi
  • Sounds very similar to Benford's law also. There seems to be a more primary factor causing the occurrence of this throughout various systems. Living in a simulation becomes more plausible the more you realize the patterns.

    Dennis GarageDennis GaragePred 5 dnevi
  • I think this all comes down to usefulness and meaning. The could be used to pinpoint any other word while sauce is so specific that it isn’t used as much because the explanation is direct and intentional. The is what exactly? but sauce is sauce.

    Lu BuLu BuPred 5 dnevi
  • 80% of my stories are filled with um and 20% of my stories are actually told

    ABC DEFABC DEFPred 5 dnevi
  • Dude, after 2 video spirals on your channel I'm not ashamed to admit that although chunks of the content my idiot brain has no way to comprehend it. But the way you present and explain the info, what I end up being able to fathom, blows my mind! Love your content.

    John HeggertyJohn HeggertyPred 5 dnevi
  • Bruh

    Shard FloofFoxShard FloofFoxPred 5 dnevi
  • We live in a simulation.

    Drooby Dooby DooDrooby Dooby DooPred 5 dnevi
  • Who else tried to count the amount of times he said "the"

    Tuckster04Tuckster04Pred 5 dnevi
  • This 80/20 split forecasted Tfues contract

    Edward TrevinoEdward TrevinoPred 6 dnevi
  • It indicates that there is one creator of the universe.. The One and Only God

    Aljulanda AlmaghadariAljulanda AlmaghadariPred 6 dnevi
  • 3:03 wow

    Kjonesey ‘Kjonesey ‘Pred 6 dnevi
  • 80 percent of my test is about 20 percent of what we had to study

    Elise VisserElise VisserPred 6 dnevi
  • Meanwhile: German *der die das die den die das die dem der dem den des der des der*

    a aa aPred 6 dnevi
  • since this cideo was published, Quizzaciously has gone from 1 result, to 17,500 results. Proving your point that the more something is used, the more it be used again.

    Ryan CooperRyan CooperPred 6 dnevi
  • 2:08 normal people: wow people really like sauce. me being a weeb: ( ͡° ͜ʖ ͡°)

    Nooby-kunNooby-kunPred 6 dnevi
  • Zipf's Law is just the algorithm that God put into a lot of shit after a long day of developing the " everything"

    Cedrick NarsisoCedrick NarsisoPred 6 dnevi
  • Vsauce always makes me feel philosophical

    toastboiitoastboiiPred 6 dnevi
  • my grandma did a funny dance on the floor and i dont know why vsauce help

    EEPred 7 dnevi
  • in the beginning was the word, and the word was with God and the word was God

    papinbalapapinbalaPred 7 dnevi
  • Is this proof that we’re in a simulation?

    James TaylorJames TaylorPred 7 dnevi
  • This law explains why you can never solve the problem of inequality in every single measurement of socio-economic status.

    pizzasmithdudepizzasmithdudePred 7 dnevi
  • I think that Zipf was correct. Speaking more than ten languages myself, I've come to the theory that language is just a way to transfer thoughts. In doing so you will need a (rare) specific word and several (common) words to describe how people should interpret the specific word.

    Mark PrinsMark PrinsPred 7 dnevi
  • "[...] in *the* english language. About one in every 16 words you encounter on a daily basis is *the* ." son of a...

    Lady YmirLady YmirPred 7 dnevi
  • Hapax Legomenon derives from the Greek Άπαξ λεγόμενον which literally means “ said once”.

    Deconstruct//RecreateDeconstruct//RecreatePred 7 dnevi
  • so then what happens when 99% of the wealth is with 1% of the people. such a huge imbalance im sure Cthulu will come and have to balance that out. or maybe thats why the sun is freaking the fuck out and is about to spit a micro nova at us.

    Brave CatBrave CatPred 7 dnevi
  • michael saying "80 percent and..." so much fucked me over ngl

    KUWALDAKUWALDAPred 7 dnevi
  • It is funny how the Pareto principle is relevant today. Here in Quebec, Canada, they said that 20% of the people at risk of contracting the COVID-19 virus (mostly people over 65 years old) represent 80% of the people hospitalized in our hospitals! I didn't know about that principle then!

    AkylliaAkylliaPred 7 dnevi
  • too bad i can't check out wordcount since flash player is dead

    QwertyQwertyPred 7 dnevi
  • Kinda puts the kibosh on the whole conservative assertion that the wealthy have what they have because they "deserve it". Seems more likely that they're just lucky beneficiaries of some cosmological constant that had little to nothing to do with their effort. This of course begs the question: if you're no more entitled to wealth or power than the next person, *why do we constantly accept systemic wealth and power inequality?*

    Mr. EMr. EPred 7 dnevi
  • "Zipf-y"

    Will It Snap?Will It Snap?Pred 8 dnevi
  • Vsauce Say "I want" and then everything after it without (The)

    Junovah HunziilJunovah HunziilPred 8 dnevi
  • What a video!!!

    Complex GodComplex GodPred 8 dnevi
  • Video must be old. The most common word now is literally :p

    Scott KeselowskiScott KeselowskiPred 8 dnevi
  • everyone use the second most used word as much as possible

    fewbronze gamesfewbronze gamesPred 8 dnevi
  • This can't be right ....

    Itchy Eyes McRealizeItchy Eyes McRealizePred 8 dnevi
  • I know use quizatiously once at least a day I no speel

    Joe BlackmoreJoe BlackmorePred 8 dnevi
  • Well order is the last thing that you'd think chaos would do, so thats exactly what it did

    The Failed_safeThe Failed_safePred 8 dnevi
  • Have you forgotten this video now that 5 years have passed?

    Hugo Le RoyHugo Le RoyPred 8 dnevi
  • He got us in the first half, not gonna lie.

    SoftAlexandra3 PlaysSoftAlexandra3 PlaysPred 8 dnevi
  • 0:26 This is a really very good Poem.

    Helpful SysopsHelpful SysopsPred 8 dnevi
SLworlds