Many scientific disciplines, especially the social sciences, face a long battle to prove that their findings represent the wider population in real world situations.
The main criteria of external validity is the process of generalization, and whether results obtained from a small sample group, often in laboratory surroundings, can be extended to make predictions about the entire population.
The reality is that if a research program has poor external validity, the results will not be taken seriously, so any research design must justify sampling and selection methods.
What is External Validity?
In 1966, Campbell and Stanley proposed the commonly accepted definition of external validity.
“External validity asks the question of generalizability: To what populations, settings, treatment variables and measurement variables can this effect be generalized?”
External validity is usually split into two distinct types, population validity and ecological validity, and they are both essential elements in judging the strength of an experimental design.
Psychology and External Validity
The Battle Lines are Drawn
External validity often causes a little friction between clinical psychologists and research psychologists.
Clinical psychologists often believe that research psychologists spend all of their time in laboratories, testing mice and humans in conditions that bear little resemblance to the outside world. They claim that the data produced has no external validity, and does not take into account the sheer complexity and individuality of the human mind.
Before we are flamed by irate research psychologists, the truth lies somewhere between the two extremes! Research psychologists find out trends and generate sweeping generalizations that predict the behavior of groups. Clinical psychologists end up picking up the pieces, and study the individuals who lie outside the predictions, hence the animosity.
In most cases, research psychology has a very high population validity, because researchers take meticulously randomly select groups and use large sample sizes, allowing meaningful statistical analysis.
However, the artificial nature of research psychology means that ecological validity is usually low.
Clinical psychologists, on the other hand, often use focused case studies, which cause minimum disruption to the subject and have strong ecological validity. However, the small sample sizes mean that the population validity is often low.
Ideally, using both approaches provides useful generalizations, over time!
Randomization in External Validity and Internal Validity
It is also important to distinguish between external and internal validity, especially with the process of randomization, which is easily misinterpreted. Random selection is an important tenet of external validity.
For example, a research design, which involves sending out survey questionnaires to students picked at random, displays more external validity than one where the questionnaires are given to friends. This is randomization to improve external validity.
Once you have a representative sample, high internal validity involves randomly assigning subjects to groups, rather than using pre-determined selection factors.
With the student example, randomly assigning the students into test groups, rather than picking pre-determined groups based upon degree type, gender, or age strengthens the internal validity.
Work Cited
Campbell, D.T., Stanley, J.C. (1966). Experimental and Quasi-Experimental Designs for Research. Skokie, Il: Rand McNally.
{"appState":{"pageLoadApiCallsStatus":true},"articleState":{"article":{"headers":{"creationTime":"2016-03-26T22:55:36+00:00","modifiedTime":"2016-03-26T22:55:36+00:00","timestamp":"2022-09-14T18:12:02+00:00"},"data":{"breadcrumbs":[{"name":"Academics & The Arts","_links":{"self":"//dummies-api.dummies.com/v2/categories/33662"},"slug":"academics-the-arts","categoryId":33662},{"name":"Math","_links":{"self":"//dummies-api.dummies.com/v2/categories/33720"},"slug":"math","categoryId":33720},{"name":"Statistics","_links":{"self":"//dummies-api.dummies.com/v2/categories/33728"},"slug":"statistics","categoryId":33728}],"title":"Generalizing Statistical Results to the Entire Population","strippedTitle":"generalizing statistical results to the entire population","slug":"generalizing-statistical-results-to-the-entire-population","canonicalUrl":"","seo":{"metaDescription":"Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is cal","noIndex":0,"noFollow":0},"content":"<p>Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is called <i>generalization,</i> and it occurs more often than you might think. People want their results instantly; they don't want to wait for them, so well-planned surveys and experiments take a back seat to instant Web surveys and convenience samples. </p>\n<p>For example, a researcher wants to know how cable news channels have influenced the way Americans get their news. He also happens to be a statistics professor at a large research institution and has 1,000 students in his classes. He decides that instead of taking a random sample of Americans, which would be difficult, time-consuming, and expensive, he will just put a question on his final exam to get his students' answers. His data analysis shows that only 5 percent of his students read the newspaper and/or watch network news programs anymore; the rest watch cable news. For his class, the ratio of students who exclusively watch cable news compared to those students who don't is 20 to 1. The professor reports this and sends out a press release about it. The cable news channels pick up on it and the next day are reporting, \"Americans choose cable news channels over newspapers and network news by a 20-to-1 margin!\"</p>\n<p>Do you see what's wrong with this picture? The problem is that the professor's conclusions go way beyond his study, which is wrong. He used the students in his statistics class to obtain the data that serves as the basis for his entire report and the resulting headline. Yet the professor reports the results about <i>all</i> Americans. It's safe to say that a sample of 1,000 college students taking a statistics class at the same time at the same college doesn't represent a cross section of America.</p>\n<p>If the professor wants to make conclusions in the end about America, he has to select a random sample of Americans to take his survey. If he uses 1,000 students from his class, then his conclusions can be made only about that class and no one else.</p>\n<p class=\"Tip\">To avoid or detect generalization, identify the population that you're intending to make conclusions about and make sure the selected sample represents that population. If the sample represents a smaller group within that population, then the conclusions have to be downsized in scope also.</p>","description":"<p>Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is called <i>generalization,</i> and it occurs more often than you might think. People want their results instantly; they don't want to wait for them, so well-planned surveys and experiments take a back seat to instant Web surveys and convenience samples. </p>\n<p>For example, a researcher wants to know how cable news channels have influenced the way Americans get their news. He also happens to be a statistics professor at a large research institution and has 1,000 students in his classes. He decides that instead of taking a random sample of Americans, which would be difficult, time-consuming, and expensive, he will just put a question on his final exam to get his students' answers. His data analysis shows that only 5 percent of his students read the newspaper and/or watch network news programs anymore; the rest watch cable news. For his class, the ratio of students who exclusively watch cable news compared to those students who don't is 20 to 1. The professor reports this and sends out a press release about it. The cable news channels pick up on it and the next day are reporting, \"Americans choose cable news channels over newspapers and network news by a 20-to-1 margin!\"</p>\n<p>Do you see what's wrong with this picture? The problem is that the professor's conclusions go way beyond his study, which is wrong. He used the students in his statistics class to obtain the data that serves as the basis for his entire report and the resulting headline. Yet the professor reports the results about <i>all</i> Americans. It's safe to say that a sample of 1,000 college students taking a statistics class at the same time at the same college doesn't represent a cross section of America.</p>\n<p>If the professor wants to make conclusions in the end about America, he has to select a random sample of Americans to take his survey. If he uses 1,000 students from his class, then his conclusions can be made only about that class and no one else.</p>\n<p class=\"Tip\">To avoid or detect generalization, identify the population that you're intending to make conclusions about and make sure the selected sample represents that population. If the sample represents a smaller group within that population, then the conclusions have to be downsized in scope also.</p>","blurb":"","authors":[],"primaryCategoryTaxonomy":{"categoryId":33728,"title":"Statistics","slug":"statistics","_links":{"self":"//dummies-api.dummies.com/v2/categories/33728"}},"secondaryCategoryTaxonomy":{"categoryId":0,"title":null,"slug":null,"_links":null},"tertiaryCategoryTaxonomy":{"categoryId":0,"title":null,"slug":null,"_links":null},"trendingArticles":null,"inThisArticle":[],"relatedArticles":{"fromBook":[],"fromCategory":[{"articleId":263501,"title":"10 Steps to a Better Math Grade with Statistics","slug":"10-steps-to-a-better-math-grade-with-statistics","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/263501"}},{"articleId":263495,"title":"Statistics and Histograms","slug":"statistics-and-histograms","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/263495"}},{"articleId":263492,"title":"What is Categorical Data and How is It Summarized?","slug":"what-is-categorical-data-and-how-is-it-summarized","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/263492"}},{"articleId":209320,"title":"Statistics II For Dummies Cheat Sheet","slug":"statistics-ii-for-dummies-cheat-sheet","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/209320"}},{"articleId":209293,"title":"SPSS For Dummies Cheat Sheet","slug":"spss-for-dummies-cheat-sheet","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/209293"}}]},"hasRelatedBookFromSearch":true,"relatedBook":{"bookId":282603,"slug":"statistics-for-dummies-2nd-edition","isbn":"9781119293521","categoryList":["academics-the-arts","math","statistics"],"amazon":{"default":"//www.amazon.com/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20","ca":"//www.amazon.ca/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20","indigo_ca":"//www.tkqlhce.com/click-9208661-13710633?url=//www.chapters.indigo.ca/en-ca/books/product/1119293529-item.html&cjsku=978111945484","gb":"//www.amazon.co.uk/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20","de":"//www.amazon.de/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20"},"image":{"src":"//catalogimages.wiley.com/images/db/jimages/9781119293521.jpg","width":250,"height":350},"title":"Statistics For Dummies","testBankPinActivationLink":"","bookOutOfPrint":true,"authorsInfo":"\n <p><p><b><b data-author-id=\"9121\">Deborah J. Rumsey</b>, PhD</b> is a longtime statistics professor at The Ohio State University specializing in statistics education. She authored <i>Statistics For Dummies, Statistics II For Dummies,</i> and <i>Probability For Dummies.</i></p>","authors":[{"authorId":9121,"name":"Deborah J. Rumsey","slug":"deborah-j-rumsey","description":" <p><b>Deborah J. Rumsey, PhD</b> is a longtime statistics professor at The Ohio State University specializing in statistics education. She authored <i>Statistics For Dummies, Statistics II For Dummies,</i> and <i>Probability For Dummies.</i> ","hasArticle":false,"_links":{"self":"//dummies-api.dummies.com/v2/authors/9121"}}],"_links":{"self":"//dummies-api.dummies.com/v2/books/282603"}},"collections":[],"articleAds":{"footerAd":"<div class=\"du-ad-region row\" id=\"article_page_adhesion_ad\"><div class=\"du-ad-unit col-md-12\" data-slot-id=\"article_page_adhesion_ad\" data-refreshed=\"false\" \r\n data-target = \"[{"key":"cat","values":["academics-the-arts","math","statistics"]},{"key":"isbn","values":[null]}]\" id=\"du-slot-6322197242359\"></div></div>","rightAd":"<div class=\"du-ad-region row\" id=\"article_page_right_ad\"><div class=\"du-ad-unit col-md-12\" data-slot-id=\"article_page_right_ad\" data-refreshed=\"false\" \r\n data-target = \"[{"key":"cat","values":["academics-the-arts","math","statistics"]},{"key":"isbn","values":[null]}]\" id=\"du-slot-6322197242c15\"></div></div>"},"articleType":{"articleType":"Articles","articleList":null,"content":null,"videoInfo":{"videoId":null,"name":null,"accountId":null,"playerId":null,"thumbnailUrl":null,"description":null,"uploadDate":null}},"sponsorship":{"sponsorshipPage":false,"backgroundImage":{"src":null,"width":0,"height":0},"brandingLine":"","brandingLink":"","brandingLogo":{"src":null,"width":0,"height":0},"sponsorAd":"","sponsorEbookTitle":"","sponsorEbookLink":"","sponsorEbookImage":{"src":null,"width":0,"height":0}},"primaryLearningPath":"Advance","lifeExpectancy":null,"lifeExpectancySetFrom":null,"dummiesForKids":"no","sponsoredContent":"no","adInfo":"","adPairKey":[]},"status":"publish","visibility":"public","articleId":201267},"articleLoadedStatus":"success"},"listState":{"list":{},"objectTitle":"","status":"initial","pageType":null,"objectId":null,"page":1,"sortField":"time","sortOrder":1,"categoriesIds":[],"articleTypes":[],"filterData":{},"filterDataLoadedStatus":"initial","pageSize":10},"adsState":{"pageScripts":{"headers":{"timestamp":"2022-10-18T10:50:01+00:00"},"adsId":0,"data":{"scripts":[{"pages":["all"],"location":"header","script":"<!--Optimizely Script-->\r\n<script src=\"//cdn.optimizely.com/js/10563184655.js\"></script>","enabled":false},{"pages":["all"],"location":"header","script":"<!-- comScore Tag -->\r\n<script>var _comscore = _comscore || [];_comscore.push({ c1: \"2\", c2: \"15097263\" });(function() {var s = document.createElement(\"script\"), el = document.getElementsByTagName(\"script\")[0]; s.async = true;s.src = (document.location.protocol == \"\" ? \"//sb\" : \"//b\") + \".scorecardresearch.com/beacon.js\";el.parentNode.insertBefore(s, el);})();</script><noscript><img src=\"//sb.scorecardresearch.com/p?c1=2&c2=15097263&cv=2.0&cj=1\" /></noscript>\r\n<!-- / comScore Tag -->","enabled":true},{"pages":["all"],"location":"footer","script":"<!--BEGIN QUALTRICS WEBSITE FEEDBACK SNIPPET-->\r\n<script type='text/javascript'>\r\n(function(){var g=function(e,h,f,g){\r\nthis.get=function(a){for(var a=a+\"=\",c=document.cookie.split(\";\"),b=0,e=c.length;b<e;b++){for(var d=c[b];\" \"==d.charAt(0);)d=d.substring(1,d.length);if(0==d.indexOf(a))return d.substring(a.length,d.length)}return null};\r\nthis.set=function(a,c){var b=\"\",b=new Date;b.setTime(b.getTime()+6048E5);b=\"; expires=\"+b.toGMTString();document.cookie=a+\"=\"+c+b+\"; path=/; \"};\r\nthis.check=function(){var a=this.get(f);if(a)a=a.split(\":\");else if(100!=e)\"v\"==h&&(e=Math.random()>=e/100?0:100),a=[h,e,0],this.set(f,a.join(\":\"));else return!0;var c=a[1];if(100==c)return!0;switch(a[0]){case \"v\":return!1;case \"r\":return c=a[2]%Math.floor(100/c),a[2]++,this.set(f,a.join(\":\")),!c}return!0};\r\nthis.go=function(){if(this.check()){var a=document.createElement(\"script\");a.type=\"text/javascript\";a.src=g;document.body&&document.body.appendChild(a)}};\r\nthis.start=function(){var t=this;\"complete\"!==document.readyState?window.addEventListener?window.addEventListener(\"load\",function(){t.go()},!1):window.attachEvent&&window.attachEvent(\"onload\",function(){t.go()}):t.go()};};\r\ntry{(new g(100,\"r\",\"QSI_S_ZN_5o5yqpvMVjgDOuN\",\"//zn5o5yqpvmvjgdoun-wiley.siteintercept.qualtrics.com/SIE/?Q_ZID=ZN_5o5yqpvMVjgDOuN\")).start()}catch(i){}})();\r\n</script><div id='ZN_5o5yqpvMVjgDOuN'><!--DO NOT REMOVE-CONTENTS PLACED HERE--></div>\r\n<!--END WEBSITE FEEDBACK SNIPPET-->","enabled":false},{"pages":["all"],"location":"header","script":"<!-- Hotjar Tracking Code for //www.dummies.com -->\r\n<script>\r\n (function(h,o,t,j,a,r){\r\n h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};\r\n h._hjSettings={hjid:257151,hjsv:6};\r\n a=o.getElementsByTagName('head')[0];\r\n r=o.createElement('script');r.async=1;\r\n r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;\r\n a.appendChild(r);\r\n })(window,document,'//static.hotjar.com/c/hotjar-','.js?sv=');\r\n</script>","enabled":false},{"pages":["article"],"location":"header","script":"<!-- //Connect Container: dummies --> <script src=\"//get.s-onetag.com/bffe21a1-6bb8-4928-9449-7beadb468dae/tag.min.js\" async defer></script>","enabled":true},{"pages":["homepage"],"location":"header","script":"<meta name=\"facebook-domain-verification\" content=\"irk8y0irxf718trg3uwwuexg6xpva0\" />","enabled":true},{"pages":["homepage","article","category","search"],"location":"footer","script":"<!-- Facebook Pixel Code -->\r\n<noscript>\r\n<img height=\"1\" width=\"1\" src=\"//www.facebook.com/tr?id=256338321977984&ev=PageView&noscript=1\"/>\r\n</noscript>\r\n<!-- End Facebook Pixel Code -->","enabled":true}]}},"pageScriptsLoadedStatus":"success"},"navigationState":{"navigationCollections":[{"collectionId":287568,"title":"BYOB (Be Your Own Boss)","hasSubCategories":false,"url":"/collection/for-the-entry-level-entrepreneur-287568"},{"collectionId":293237,"title":"Be a Rad Dad","hasSubCategories":false,"url":"/collection/be-the-best-dad-293237"},{"collectionId":294090,"title":"Contemplating the Cosmos","hasSubCategories":false,"url":"/collection/theres-something-about-space-294090"},{"collectionId":287563,"title":"For Those Seeking Peace of Mind","hasSubCategories":false,"url":"/collection/for-those-seeking-peace-of-mind-287563"},{"collectionId":287570,"title":"For the Aspiring Aficionado","hasSubCategories":false,"url":"/collection/for-the-bougielicious-287570"},{"collectionId":291903,"title":"For the Budding Cannabis Enthusiast","hasSubCategories":false,"url":"/collection/for-the-budding-cannabis-enthusiast-291903"},{"collectionId":291934,"title":"For the Exam-Season Crammer","hasSubCategories":false,"url":"/collection/for-the-exam-season-crammer-291934"},{"collectionId":287569,"title":"For the Hopeless Romantic","hasSubCategories":false,"url":"/collection/for-the-hopeless-romantic-287569"},{"collectionId":287567,"title":"For the Unabashed Hippie","hasSubCategories":false,"url":"/collection/for-the-unabashed-hippie-287567"},{"collectionId":292186,"title":"Just DIY It","hasSubCategories":false,"url":"/collection/just-diy-it-292186"}],"navigationCollectionsLoadedStatus":"success","navigationCategories":{"books":{"0":{"data":[{"categoryId":33512,"title":"Technology","hasSubCategories":true,"url":"/category/books/technology-33512"},{"categoryId":33662,"title":"Academics & The Arts","hasSubCategories":true,"url":"/category/books/academics-the-arts-33662"},{"categoryId":33809,"title":"Home, Auto, & Hobbies","hasSubCategories":true,"url":"/category/books/home-auto-hobbies-33809"},{"categoryId":34038,"title":"Body, Mind, & Spirit","hasSubCategories":true,"url":"/category/books/body-mind-spirit-34038"},{"categoryId":34224,"title":"Business, Careers, & Money","hasSubCategories":true,"url":"/category/books/business-careers-money-34224"}],"breadcrumbs":[],"categoryTitle":"Level 0 Category","mainCategoryUrl":"/category/books/level-0-category-0"}},"articles":{"0":{"data":[{"categoryId":33512,"title":"Technology","hasSubCategories":true,"url":"/category/articles/technology-33512"},{"categoryId":33662,"title":"Academics & The Arts","hasSubCategories":true,"url":"/category/articles/academics-the-arts-33662"},{"categoryId":33809,"title":"Home, Auto, & Hobbies","hasSubCategories":true,"url":"/category/articles/home-auto-hobbies-33809"},{"categoryId":34038,"title":"Body, Mind, & Spirit","hasSubCategories":true,"url":"/category/articles/body-mind-spirit-34038"},{"categoryId":34224,"title":"Business, Careers, & Money","hasSubCategories":true,"url":"/category/articles/business-careers-money-34224"}],"breadcrumbs":[],"categoryTitle":"Level 0 Category","mainCategoryUrl":"/category/articles/level-0-category-0"}}},"navigationCategoriesLoadedStatus":"success"},"searchState":{"searchList":[],"searchStatus":"initial","relatedArticlesList":[],"relatedArticlesStatus":"initial"},"routeState":{"name":"Article3","path":"/article/academics-the-arts/math/statistics/generalizing-statistical-results-to-the-entire-population-201267/","hash":"","query":{},"params":{"category1":"academics-the-arts","category2":"math","category3":"statistics","article":"generalizing-statistical-results-to-the-entire-population-201267"},"fullPath":"/article/academics-the-arts/math/statistics/generalizing-statistical-results-to-the-entire-population-201267/","meta":{"routeType":"article","breadcrumbInfo":{"suffix":"Articles","baseRoute":"/category/articles"},"prerenderWithAsyncData":true},"from":{"name":null,"path":"/","hash":"","query":{},"params":{},"fullPath":"/","meta":{}}},"dropsState":{"submitEmailResponse":false,"status":"initial"},"sfmcState":{"status":"initial"},"profileState":{"auth":{},"userOptions":{},"status":"success"}}
Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is called generalization, and it occurs more often than you might think. People want their results instantly; they don't want to wait for them, so well-planned surveys and experiments take a back seat to instant Web surveys and convenience samples. For example, a researcher wants to know how cable news channels have influenced the way Americans get their news. He also happens to be a statistics professor at a large research institution and has 1,000 students in his classes. He decides that instead of taking a random sample of Americans, which would be difficult, time-consuming, and expensive, he will just put a question on his final exam to get his students' answers. His data analysis shows that only 5 percent of his students read the newspaper and/or watch network news programs anymore; the rest watch cable news. For his class, the ratio of students who exclusively watch cable news compared to those students who don't is 20 to 1. The professor reports this and sends out a press release about it. The cable news channels pick up on it and the next day are reporting, "Americans choose cable news channels over newspapers and network news by a 20-to-1 margin!" Do you see what's wrong with this picture? The problem is that the professor's conclusions go way beyond his study, which is wrong. He used the students in his statistics class to obtain the data that serves as the basis for his entire report and the resulting headline. Yet the professor reports the results about all Americans. It's safe to say that a sample of 1,000 college students taking a statistics class at the same time at the same college doesn't represent a cross section of America. If the professor wants to make conclusions in the end about America, he has to select a random sample of Americans to take his survey. If he uses 1,000 students from his class, then his conclusions can be made only about that class and no one else. To avoid or detect generalization, identify the population that you're intending to make conclusions about and make sure the selected sample represents that population. If the sample represents a smaller group within that population, then the conclusions have to be downsized in scope also.