The Art Of Statistical Thinking Detect Misinformation, Understand The World Deeper, And Make Better Decisions. By Albert Rutherford And Jae H. Kim, PhD AudioChapter
Hear it Here - https://adbl.co/3WjBwT0
00:00:00 The art of statistical thinking
00:03:31 1. Sample versus population
00:06:57 2. Descriptive statistics
00:15:42 3. Sample statistics and population parameters
00:19:26 4. Descriptive statistics for relative position
00:24:09 5. Data Visualization
00:26:25 6. Comparing alternative distributions
00:30:21 7. Normal distribution
00:34:32 8. Checking the normality of a distribution
00:38:03 9. Concluding remarks
https://www.amazon.com/dp/B0BJ7QGMBL
Not knowing statistics can lead to a loss of money, time, and accurate information.
What am I looking at? What do these numbers mean? Why? These are frequent thoughts of those who don’t know much about statistics.
“I’m not a number’s person” is not a good excuse to avoid learning the basics of this essential skill. Are you a person who earns money? Do you shop at the supermarket? Do you vote? Do you read the news? I’m sure you do.
Learn to make decisions like world leaders do.
Do you like to make uninformed, often poor decisions? Are you okay with being manipulated by skewed charts and diagrams? How about being lied to about the effectiveness of a product? I’m sure you don’t.
Statistics can help you make exponentially better calls on what to buy, who to listen to, and what to believe.
This book offers a detailed, illustrated breakdown of the fundamentals of statistics. Develop and use formal logical thinking abilities to understand the message behind numbers and charts in science, politics, and economy.
Sharpen your critical and analytic thinking skills.
Know what to look for when analyzing data. Information gets skewed – often unintentionally – because of the mainstream ways of doing statistics that didn’t catch up to big data. Stop staying in the dark. This book shines the light on the most common statistical methods - and their most frequent misuse. This step-by-step guide not only helps you detect what goes wrong in statistics but also educates you on how to utilize invaluable information statistics gets right to your benefit.
Avoid making decisions on misleading information.
- How to Use Descriptive and Inferential Statistics to Understand the World.
- Be Wary of Misleading Charts.
- Make Better Decisions Using Probability.
- Understand P-Values in Research.
- Understand Potential Bias in Studies.
Albert Rutherford is the internationally bestselling author of several books on systems thinking, game theory, and mathematical thinking. Jae H. Kim is a freelance writer in econometrics, statistics, and data analysis. Since obtaining his PhD in econometrics in 1997, he has been a professor in major Australian universities until 2022. He has published more than 70 academic articles and book chapters in econometrics, empirical finance, economics, and applied statistics, which have attracted nearly 5000 citations to date.
Learn basic statistics and spend your money wisely.
Statistics, as a learning tool, can be used or misused. Some will actively lie and mislead with statistics. More often, however, well-meaning people – even professionals - unintentionally report incorrect statistical conclusions. Knowing what errors and mistakes to look for will help you to be in a better position to evaluate the information you have been given.
#Data #DataVisualization #Descriptive #Deviation #Histogram #Interquartile #IQ #Mean #Median #MedianStandard #Outlier #Percentile #Percentiles #Quartile #StandardDeviation #StatisticalThinking #Variance #Visualization #RussellNewton #NewtonMG #TheArtofStatisticalThinking #DetectMisinformation #UnderstandTheWorldDeeper #AndMakeBetterDecisions. #ByAlbertRutherfordAndJaeH.Kim #PhD #AlbertRutherford # #JaeH.KimPhD
Transcript
The art of statistical thinking, detect misinformation, understand the world
Speaker:deeper, and make better decisions. Advanced Thinking Skills, book 3, written by
Speaker:Albert Rutherford, J. H. Kim, Ph.D., narrated by Russell Newton.
Speaker:We make decisions every day - some can change our lives and those of our loved ones.
Speaker:But it is not only the individuals who make decisions.
Speaker:Companies, courts of law, governments and international organizations also make decisions,
Speaker:often on a large scale, that can affect our jobs, the justice system, and everyday life
Speaker:in a positive or negative way.
Speaker:Such decisions usually are made under incomplete information and uncertainty.
Speaker:The decision-makers often make correct decisions that will benefit our society, but they make
Speaker:incorrect decisions too.
Speaker:The cost of the latter can sometimes be devastating, starting from personal tragedies to changing
Speaker:the course of human history.
Speaker:But let’s not run so far ahead.
Speaker:Suppose you are making an investment decision for your retirement.
Speaker:Investment funds report their average returns for the past 5 years; you read a media report
Speaker:about the recent growth of the real estate market, and you hear about overnight millionaires
Speaker:who have made big from investing in cryptocurrency.
Speaker:You also hear about those who lost their life savings because of wrong investments or scams.
Speaker:And there is always a catch in the fine print - “Past performance is not necessarily indicative
Speaker:of future performance."
Speaker:This means you are facing uncertainty in your investment decisions, and you should learn
Speaker:how to make a well-informed decision under this circumstance.
Speaker:If you make a decision after you sampled a range of different funds, compared them with
Speaker:those of real estate markets, and studied the future prospect of the world economy,
Speaker:learned from the investment gurus such as Warren Buffet and listened to your friends
Speaker:and advisors, then it is most likely that you have made an informed decision that will
Speaker:bring handsome payoff eventually.
Speaker:This is, in a way, “statistical thinking”; you sample the population and learn from it
Speaker:to make an informed decision.
Speaker:The more diverse and informative your sample’s elements are, the more likely it is that you
Speaker:have made the right decision.
Speaker:This book will show you how to understand statistics as a layman and make informed decisions
Speaker:with the help of statistical thinking.
Speaker:The problem is that statistics can easily be manipulated and misinterpreted.
Speaker:If statistical findings were always presented and utilized in an honest and correct way,
Speaker:the results wouldn’t always be as rosy.
Speaker:We often see distorted and misguided numbers and outcomes, even though that was not the
Speaker:intention of those who report statistics.
Speaker:This book is intended to help readers gain better understanding and decision-making skills
Speaker:– the kind that professional statisticians possess.
Speaker:In the first chapter, we will review the definitions and basic concepts of statistics.
Speaker:As a book on statistics, it is inevitable to introduce mathematical details.
Speaker:However, these details will only be presented when necessary, without providing the full
Speaker:theoretical background.
Speaker:Chapter 1 - Definition and Basic Concepts.
Speaker:1.
Speaker:Sample versus population.
Speaker:An investor wishes to know the five-year average return from investing in the U. S. stock market.
Speaker:There are nearly 2,400 stocks (as of August 2022) listed on the NYSE (New York Stock Exchange),
Speaker:and they must select a manageable number of stocks to form a portfolio of stocks.
Speaker:However, they don’t need to calculate the average return of all 2400 stocks.
Speaker:There are stocks not worth investing in – too low return or too risky.
Speaker:Our investor will need to select a set of stocks that suits their investment style.
Speaker:In this example, the collection of all stocks in the NYSE is called the population in statistical
Speaker:jargon, and a subset of all stocks is called a sample.
Speaker:Collecting the information from all the members of the population is too costly and time-consuming
Speaker:and even unnecessary.
Speaker:We can obtain a good indicator of average return by looking at a sample.
Speaker:The way we select the sample is critically important, and it depends largely on the purpose
Speaker:of the study or the aim of the statistical task at hand.
Speaker:Suppose the investor’s aim is to achieve a steady return with relatively low risk by
Speaker:investing in big and stable companies.
Speaker:Then a good sample is the Dow Jones index, which comprises the stocks of 30 prominent
Speaker:companies, such as Boeing, Coca-Cola, Microsoft, and Proctor & Gamble.
Speaker:If the investor’s goal is to achieve a higher return with higher growth, albeit taking a
Speaker:higher risk, the NASDAQ-100 index is a good sample that mainly includes the top technology
Speaker:and IT stocks, such as Amazon, Apple, eBay, and Google.
Speaker:By looking at the average returns of these indices, the investor can get a clear indication
Speaker:and impression of the performance of these stocks.
Speaker:Seasoned investors can select their own sample based on their aim and risk-return preference.
Speaker:The important point is that the sample should be a good representation of the target population.
Speaker:If the investor wants safe and steady investment returns, but their sample represents high-risk
Speaker:stocks, they may not effectively achieve the aim of their investment.
Speaker:Hence, the target population should be determined in consideration of the aim of the statistical
Speaker:study.
Speaker:A sample that is a good representation of the population can be obtained by pure random
Speaker:sampling.
Speaker:The members of the population are selected randomly with an equal chance.
Speaker:For example, in political polls, all eligible voters should be treated equally.
Speaker:In this situation, the most effective way of selecting an unbiased and representative
Speaker:sample is random sampling, where the members of the eligible voters are selected with equal
Speaker:chance, with no pre-selection or exclusions.
Speaker:In a later chapter, we will discuss an example of one of the most disastrous polling outcomes
Speaker:in the history, which occurred due to a violation of this random sampling principle.
Speaker:2.
Speaker:Descriptive statistics.
Speaker:Descriptive statistics is a branch of statistics where the sample features are presented with
Speaker:a range of summary statistics and visualization methods.
Speaker:The summary statistics include the mean and median, which describe the centre of the sample
Speaker:values, and the variance and standard deviation are the measures of the variability of the
Speaker:sample values.
Speaker:Visualization methods include plots, charts, and graphs, which are used to make a visual
Speaker:impression about the distribution of the sample values.
Speaker:1.1.
Speaker:Mean and median.
Speaker:The mean refers to the average of a set of values.
Speaker:It is computed by adding the numbers and dividing the total by the number of observations.
Speaker:The mean is the average of the sample values of size n, with each individual point given
Speaker:the weight of 1/n.
Speaker:The formula for the mean can be written as, .
Speaker:(1).
Speaker:where (X1, X2,…, Xn) represent the data points and n is called the sample size.
Speaker:That is, the sample mean is the sum of all sample points divided by the sample size.
Speaker:Alternatively, it can be interpreted as a weighted sum of all data points with an equal
Speaker:weight of 1/n.
Speaker:The median is the middle number in a sequence of numbers.
Speaker:To find the median, organize each number in order by size; the number in the middle is
Speaker:the median.i In statistical terms, the median is defined as the middle value of (X1, X2,
Speaker:…, Xn) when sorted in ascending or descending order.
Speaker:Consider a simple example of (X1, …, Xn) = (1, 2, 3, 4, 5) and n = 5.
Speaker:The sum of all X’s is 15 (1+2+3+4+5=15), and the sample mean is 3 (15/5=3).
Speaker:The middle value of (1, 2, 3, 4, 5) is 3.
Speaker:In this case, the sample’s mean and median are the same.
Speaker:In general, the mean and median values are different, and the median is widely used where
Speaker:there are possible extreme values in the sample points.
Speaker:Consider the sample points with an extreme observation (X1, …, Xn) = (1, 2, 3, 4, 20),
Speaker:then the sample mean is 6 (1+2+3+4+20 = 30; 30/5=6), and the median is still 3 as the
Speaker:middle value of the distribution (1, 2, 3, 4, 20).
Speaker:If this extreme value is unusual and does not represent the target population, then
Speaker:the sample mean of 6 can be a misleading value because it was distorted by the presence of
Speaker:20.
Speaker:In this case, the median should be preferred to the mean.
Speaker:A practical example of using the median over the mean is the case for house prices.
Speaker:For example, the researcher is interested in the average house price in a middle-class
Speaker:suburb.
Speaker:In such a suburb, there is still a chance that a big mansion or two in a large block
Speaker:of land may be included in the sale.
Speaker:However, these houses do not represent the general characteristics of the suburb, and
Speaker:it is reasonable to use the median in this case to find the average value free from the
Speaker:effect of these extreme values1.
Speaker:The mean vs. median is closely related with the “skewedness” of the distribution.
Speaker:If the distribution of the numbers you have is (more or less) symmetric around the mean
Speaker:as in (X1, …, Xn) = (1, 2, 3, 4, 5), the mean and median will be identical or practically
Speaker:the same.
Speaker:However, when the distribution of the numbers is asymmetric or skewed, then the mean and
Speaker:median can be different.
Speaker:For example, if the distribution is asymmetric, as in (X1, …, Xn) = (1, 2, 3, 4, 20), then
Speaker:the two values can be different.
Speaker:Photo source - Study.comii.
Speaker:Graphical illustrations of the different shapes of the distribution and the positions of the
Speaker:mean and median are given above.
Speaker:Suppose the above is the distribution of the performance of all salespeople in a company.
Speaker:A symmetric distribution means the higher performers and lower performers are in the
Speaker:same or similar proportion; in which case the mean and median are almost identical.
Speaker:A positive skewed distribution means the presence of a small number of extremely capable performers.
Speaker:In this case, the mean of the sales is inflated by their performance.
Speaker:If the sales manager wants an average value that represents the performance of the “average
Speaker:salesperson”, then the use of median is appropriate.
Speaker:If she wants to know the average sales, including the performance of all salespeople in the
Speaker:company, then the use of the mean is appropriate.
Speaker:A similar interpretation can also be made from a negatively skewed distribution illustrated
Speaker:above.
Speaker:1.2.
Speaker:Variance and standard deviation.
Speaker:When analyzing or presenting a set of numbers, it is important to know the centre of the
Speaker:distribution.
Speaker:But understanding their dispersion and variability is also important.
Speaker:Consider two salespeople with the same or a similar number of mean sales in the past
Speaker:year.
Speaker:In evaluating who was a more consistent performer, the manager will compare the dispersions in
Speaker:their sales throughout the year.
Speaker:Measures of variability, variance, and standard deviation present how widespread the sample
Speaker:points are around the mean.
Speaker:The distance of the sample point from the mean is calculated as , and they are squared
Speaker:to make them all positive.
Speaker:The average of all the squared distances from the mean is called the variance, which can
Speaker:be written as,.
Speaker:(0).
Speaker:How this formula works will be explained in the table below.
Speaker:But it is, in a way, the average of the squared distance of the data points from the mean,
Speaker:i.e., .
Speaker:The standard deviation (s) is defined as the square root of the variance, namely, .
Speaker:(0).
Speaker:Since the variance is the distance of the sample points from the mean in squares, the
Speaker:standard deviation converts the value into the same unit as the original value of the
Speaker:sample points by taking the square root.
Speaker:X.
Speaker:1.
Speaker:-2 (=1-3).
Speaker:-22 = 4.
Speaker:2.
Speaker:-1(=2-3).
Speaker:-22 = 1.
Speaker:3.
Speaker:0 (=3-3).
Speaker:02 = 0.
Speaker:4.
Speaker:1 (=4-3).
Speaker:12 = 1.
Speaker:5.
Speaker:2 (=1-3).
Speaker:22 = 4.
Speaker:Sum.
Speaker:10.
Speaker:=3.
Speaker:Using the example we used above as an illustration, X= (1, 2, 3, 4, 5) and The variance is the
Speaker:sum of the numbers in the last column on the chart above divided by 4, which is 10/4 = 2.5.
Speaker:The standard deviation is .
Speaker:The interpretation is that the sample points are, on average, 1.58 units away from the
Speaker:mean value of 3.
Speaker:Why the division (or weight) is by (n-1), not by n, is beyond the scope of this book,
Speaker:but it is to make the calculation more accurate when the sample size is small.
Speaker:When the sample size is large, the division by n or by (n-1) makes no practical difference.
Speaker:There are other variability measures around the median (i.e., interquartile range), and
Speaker:they will be introduced in this book later.
Speaker:3.
Speaker:Sample statistics and population parameters.
Speaker:The sample mean () and standard deviation (s) are the statistics calculated from a sample.
Speaker:The sample is a subset of the population, which also has the mean and standard deviation
Speaker:(the median and variance as well).
Speaker:When we use statistics, what we eventually want to know is the population values (also
Speaker:called the population parameters), such as the mean and standard deviation.
Speaker:The population mean and standard deviation are often written with Greek letters as and
Speaker:, values that are never known.
Speaker:Suppose you want to know the mean household income of California.
Speaker:If you visit all the households in California to find their mean income, as in a census,
Speaker:you are looking for the value of .
Speaker:However, such an exercise is often neither feasible nor necessary.
Speaker:A good representative sample can tell us a lot about , as we shall see later.
Speaker:We can gather a random sample of 1,000 households to find their income, and this will give the
Speaker:value of the sample mean ().
Speaker:If the sample was a good representation of the population, it is likely the sample mean
Speaker:is a good indicator for the population mean.
Speaker:The population and variance (and standard deviation) can be written formally as,.
Speaker:(0).
Speaker:(0).
Speaker:(0).
Speaker:where north is the population size and represent the population values.
Speaker:The formulae above are similar to their sample counterparts in (1) to (3), hence their interpretations
Speaker:are similar, but they are the values of the population.
Speaker:In our example, north is the number of the total households in California, and are their
Speaker:incomes.
Speaker:If 1,000 households are selected randomly and their mean income is found to be $75,000,
Speaker:then with n =1,000.
Speaker:It is hoped that this value of the sample mean is in close neighbourhood of the true
Speaker:value of the population mean.
Speaker:Let us take another example.
Speaker:Consider a fictitious country with 1 million (north) eligible voters who are voting for
Speaker:their President.
Speaker:A candidate should have the support rate of more than 0.5 to get elected.
Speaker:The true value of the support rate () is unknown, and what matter is this value on
Speaker:the election date.
Speaker:A poll is conducted from a sample of 1000 (n) eligible voters, 10 days before the election
Speaker:date.
Speaker:This value is the sample mean ().
Speaker:Suppose this sample value () is 50.1 per cent.
Speaker:This value is called an estimate of the population parameter ().
Speaker:If the sample is a good representation of the population, this estimate of sample mean
Speaker:is an indicator for the value of , 10 days before the election date.
Speaker:4.
Speaker:Descriptive statistics for relative position.
Speaker:Suppose your IQ score is 115.
Speaker:A natural question is how smart are you (according to the IQ score only) relative to the other
Speaker:people in the sample or population.
Speaker:Suppose your annual income is $50,000.
Speaker:You want to know how rich or how poor you are relative to the others in the sample or
Speaker:population.
Speaker:You ran a marathon, and you completed the race with a record of 3 hours.
Speaker:You want to know your rank in the race and where your rank stands relative to all the
Speaker:participants of the race.
Speaker:These questions are asking for a relative position, another important question in statistics.
Speaker:The popular measures of relative positions are percentiles (sometimes called quantiles)
Speaker:and quartiles.
Speaker:Percentiles (quantiles).
Speaker:With percentiles, we divide the distribution of the numbers into 100 positions.
Speaker:For example, the 90th percentile represents the value in the sample that has 10% of the
Speaker:sample points higher and 90% of the values lower than it.
Speaker:That is, if your IQ score of 115 is said to be the 90th percentile, this means you are
Speaker:at the top 10% of the distribution of all IQ scores.
Speaker:Suppose your income of $50,000 is the 40th percentile of the distribution, then it means
Speaker:your income is at the bottom 40% of the distribution.
Speaker:That is, if there were 1000 people in the sample, your income stands at the 400th position
Speaker:when all incomes are sorted in ascending order.
Speaker:Similarly, among the 100 runners who participated in the marathon event, suppose your record
Speaker:of 3 hours is at the 75th percentile.
Speaker:This means your record is at the top 25%, and there are 24 runners who finish the race
Speaker:with a better record than yours, and 74 of them were behind you.
Speaker:Quartiles.
Speaker:Quartiles are similar to percentile, but instead of dividing the distribution of the numbers
Speaker:into 100 positions, they are based on the division into 4, as the following table shows
Speaker:- .
Speaker:The first quartile is the value whose position is at the bottom 25%, and it is the same as
Speaker:the 25th percentile.
Speaker:The second quartile is the 50th percentile, which is also the median.
Speaker:If we go back to your marathon record, your record of 3 hours is the third quartile of
Speaker:the distribution.
Speaker:Interquartile range.
Speaker:An interquartile range is defined as the difference between the third and 1st quartile of the
Speaker:distribution.
Speaker:It is a measure of variability or dispersion of a distribution alternative to the standard
Speaker:deviation.
Speaker:As the difference between the 3rd and 1st quartiles, the length of the interval contains
Speaker:the (middle) 50% of the data points around the median.
Speaker:Similarly to the median, the interquartile range is not sensitive to a few extreme values
Speaker:in the distribution, while standard deviation can be inflated by extreme values.
Speaker:More examples will follow for the interquartile range.
Speaker:As an example, consider two suburbs whose median house prices are similar at 1 million
Speaker:dollars.
Speaker:The researcher finds the first suburb has the 1st quartile at the $750,000 and the 3rd
Speaker:quartile at $1.25 million, with the interquartile range of $500,000 ($1.25 million - $750,000).
Speaker:The second suburb has the 1st quartile at the $500,000 and the 3rd quartile at $1.5
Speaker:million, with the interquartile range of 1 million dollars ($1.5 million – $500,000).
Speaker:The interval that contains the middle 50% of the house prices are much longer in the
Speaker:second suburb, which indicates the variability of house prices is substantially larger in
Speaker:the second suburb.
Speaker:5.
Speaker:Data Visualization.
Speaker:Visualization is a powerful way of understanding the key features of a sample and making impressions.
Speaker:It often makes a better and stronger impression about the data characteristics than a table
Speaker:full of numbers.
Speaker:Consider an investor who wishes to invest in U. S. stocks.
Speaker:They gather the sample for NASDAQ-100 index and want to know how the index and its return
Speaker:have performed in the last 5 years to December 2021.
Speaker:Figure 1 presents the line charts (time plots) of and return (growth rate) in percentage,
Speaker:monthly from 2017 to 2021.
Speaker:The index has been growing with an upward trend for the last 5 years, and the trend
Speaker:gets steeper from early 2020.
Speaker:The monthly return fluctuates around 0, with most values between -10% and 10%.
Speaker:These plots provide a clear impression of how the index has performed in the last five
Speaker:years.
Speaker:Figure 1 - Time plots of NASDAQ-100 index and return.
Speaker:Data source - Yahoo Finance.
Speaker:A histogram is another popular method of data visualization that presents the frequencies
Speaker:of data points over the intervals of sample points.
Speaker:It is a useful method of presenting the distributional shape of the sample points.
Speaker:Figure 2 presents the histogram of the monthly returns, which shows the monthly returns are
Speaker:centred between 0% and 5%, and most of the values are in the range of -10% and 10%.
Speaker:The sample mean value of the monthly return is 2.02%, and their median is 2.68%, so the
Speaker:index has been increasing at an average growth rate of just higher than 2%.
Speaker:The standard deviation is 4.92%, which indicates the average deviation of the monthly returns
Speaker:from the mean has been around 5%.
Speaker:By combining the plots and summary statistics, the investor can learn about the performance
Speaker:of the index in detail.
Speaker:Figure 2 - Histogram of Returns from NASAQQ-100 index.
Speaker:Data source - Yahoo Finance.
Speaker:6.
Speaker:Comparing alternative distributions.
Speaker:Now suppose the investor wishes to compare the performance of the NASDAQ-100 with the
Speaker:Apple stock (APPL) for the same period.
Speaker:The following table compares the basic statistics discussed so far.
Speaker:Monthly returns for two alternative investments.
Speaker:NASDAQ-100.
Speaker:APPL.
Speaker:Mean.
Speaker:2.01.
Speaker:3.02.
Speaker:Median.
Speaker:2.68.
Speaker:5.00.
Speaker:Standard Deviation.
Speaker:4.92.
Speaker:8.34.
Speaker:1st Quartile.
Speaker:-0.18.
Speaker:-1.66.
Speaker:3rd Quartile.
Speaker:5.13.
Speaker:9.25.
Speaker:10th percentile.
Speaker:-5.89.
Speaker:-7.35.
Speaker:90th percentile.
Speaker:7.37.
Speaker:12.27.
Speaker:Data source - Yahoo finance.
Speaker:The figures in this table reveal many details of the two investment alternatives -
Speaker:•The average return from NASDAQ-100 is substantially lower than APPL. The mean and median of the
Speaker:former is 2.01% and 2.68% per month, but those of APPL 3.02% and 5.00%.
Speaker:•For both cases, the median is larger than the mean, especially the APPL. This means
Speaker:the distribution is skewed to the left, with the presence of extremely low returns.
Speaker:This means, when they go down, they can go down deep!
Speaker:(Especially APPL!).
Speaker:•The variability is a lot higher for the returns from APPL. The standard deviation
Speaker:of APPL (8.34) is nearly twice larger than that of NASDA-100 (4.92).
Speaker:This means APPL has a lot larger variation around the mean.
Speaker:•The interquartile range for APPL is 10.91 (9.25 + 1.66) and that of NASDAQ-100 is 5.31
Speaker:(5.13+0.18).
Speaker:The length of interval that contains the middle 50% of the returns around the median is again
Speaker:nearly twice larger for the APPL. .
Speaker:•The worst possible outcome with 10% chance for APPL has been -7.35%, and that for NASDAQ-100
Speaker:has been -5.89%.
Speaker:The best possible outcome with 10% chance for APPL has been 12.27% a month, and that
Speaker:for NASDAQ-100 has been 7.37%.
Speaker:The comparison of these descriptive statistics reveals that monthly returns are a lot higher
Speaker:for APPL investment, but it shows substantially higher variability or risk.
Speaker:This is a well-known principle in finance - a higher return is compensation for taking
Speaker:a higher risk.
Speaker:The above plots present the histograms for the two investments.
Speaker:A larger variability of the APPL with a heavier skew to the left of the distribution than
Speaker:NASDAQ-100 is clear.
Speaker:While the summary statistics tell the difference with the numbers, these histograms can make
Speaker:a visual comparison.
Speaker:To make a further visual comparison, another method of visualisation called the “Box-Whisker”
Speaker:plot is introduced.
Speaker:It plots the mean, the median, the 1st quartile, the 3rd quartile, maximum and minimum, along
Speaker:with outliers.
Speaker:The box in the middle is based on the 3rd quartile and 1st quartile, and the height
Speaker:of the box represents the interquartile range.
Speaker:Outliers are determined by a certain criterion (i.e., the outliers are defined as those lying
Speaker:three standard deviations away from the mean).
Speaker:Again, the APPL investment gives a substantially higher median return per month, but its monthly
Speaker:variability is much higher than NASDAQ-100.
Speaker:Which investment to choose depends on how risk-averse or risk-tolerant the investor
Speaker:is.
Speaker:If you are a Braveheart and enjoy a roller coaster ride, investing in APPL is not a bad
Speaker:choice; otherwise, stick to the NASDAQ-100 for a safer option.
Speaker:7.
Speaker:Normal distribution.
Speaker:Figure 2 presents a distribution of the sample points using a histogram.
Speaker:In statistics, distribution is an important feature for both the sample and the population.
Speaker:While we can observe a distribution of the sample as in Figure 2, that of the population
Speaker:is often unknown and not observable.
Speaker:Understanding the features of a distribution is one of the fundamental questions of statistics.
Speaker:For example, what is the chance that investing in the NASDAQ-100 index will provide a return
Speaker:greater than 2%?
Speaker:What proportion of the households in California has a lower annual income than $50,000?
Speaker:We can only guess using the distribution of the sample we observe.
Speaker:Again, if the sample is a fair representation of the population, the distribution of the
Speaker:sample can well reflect the distribution of the population.
Speaker:On the other hand, there are several known distributions in statistics where the probability
Speaker:can be calculated using the given values of the parameters, such as the mean and standard
Speaker:deviation.
Speaker:Among them, the most fundamental and popular is the normal distribution.
Speaker:It is also a key distribution in the inferential statistics to be discussed in the next chapter.
Speaker:Normal distribution is a bell-shaped distribution, symmetric around its mean (or median), and
Speaker:the probability at any point of the distribution is known.
Speaker:A normal distribution with a mean and standard deviation of is written as north(,).
Speaker:In the special case of the mean being zero and the standard deviation 1, it is called
Speaker:standard normal distribution, and it is denoted as north(0,1).
Speaker:Figure 3 is a screenshot from an online calculator.2 .
Speaker:Figure 3 - Standard normal distribution.
Speaker:Given the values of the mean and standard deviation, any probability between an interval
Speaker:can be calculated.
Speaker:Figure 3 shows a normal distribution with zero mean and a standard deviation of 1 (called
Speaker:the standard normal distribution).
Speaker:Suppose your return (in percentage) from an investment follows the standard normal distribution.
Speaker:The probability that your return is between -1.96% and 1.96% is calculated to be 0.95
Speaker:(dark area on the bell illustration).
Speaker:This also means the probability of the tail areas is 5% (white area on the bell illustration).
Speaker:Your investment return can be lower than -1.96% with the probability of 0.025 and can take
Speaker:a value greater than 1.96% with the probability 0.025.
Speaker:Let’s assume the household income in California follows a normal distribution of $75,000 with
Speaker:the standard deviation of $30,000 (see Figure 4).
Speaker:Then, the household income distribution of California is represented by the bell curve
Speaker:in Figure 4.
Speaker:The probability that a household income is less than $50,000 or the proportion of the
Speaker:households with income less than $50,000 is represented by the dark area in the distribution,
Speaker:which is 0.20 approximately.
Speaker:In other words, if you pick a household at random, you have a 0.20 chance to bump into
Speaker:one with an income less than $50,000.
Speaker:This also means the chance of a randomly selected household having an income higher than $50,000
Speaker:is around 0.80 (= 1- 0.2023) .
Speaker:Figure 4 - Application of a normal distribution.
Speaker:8.
Speaker:Checking the normality of a distribution.
Speaker:Normal distribution is the most fundamental and popular distribution in statistics, and
Speaker:it is widely used as a “benchmark” distribution or as an “approximation” to the true distribution
Speaker:when it is unknown.
Speaker:Being a benchmark or approximation means it may be sometimes useful, but sometimes not,
Speaker:depending on the context and situation.
Speaker:Figure 5 is the histogram we have seen in Figure 2, the returns from NASDAQ-100 investment,
Speaker:overlayed with the normal distribution with the same mean and standard deviation values
Speaker:of the returns.
Speaker:While the histogram shows a similar shape to the normal distribution, with near symmetry
Speaker:and bell curve, the fine details are not impressively consistent with the normal distribution.
Speaker:While an approximation by a normal distribution to a stock return distribution is sometimes
Speaker:used, it is generally accepted that a stock return distribution shows a clear departure
Speaker:from a normal distribution.
Speaker:Figure 5 - Histogram of the NASDAQ-100 and APPL returns and a normal curve.
Speaker:The Q-Q (quantile-quantile) plot provides a clearer way of checking the normality of
Speaker:a sample distribution using a graphical method.
Speaker:It connects the sample quantiles (or percentiles) with the (theoretical) quantiles from the
Speaker:normal distribution.
Speaker:If the sample follows the standard normal distribution, then its percentiles should
Speaker:match the percentiles from the normal distribution with the same mean and standard deviation.
Speaker:The 95th percentile from the sample distribution (which is normal) should match the 1.96, and
Speaker:the 50th percentile from the sample distribution should be 0, which is the 50th percentile
Speaker:from the normal distribution.
Speaker:An example of the Q-Q plot is given here - .
Speaker:The grid lines are at (-1.96, 0, 1.96) for both axes, which are the 2.5th, 50th, and
Speaker:97.5th percentiles from the standard normal distribution.
Speaker:The y-axis (vertical) represents the sample quantile, and the x-axis (horizontal) represents
Speaker:the theoretical quantiles from the normal distribution.
Speaker:The grid lines are at (-1.96, 0, 1.96) for both axes, which match exactly.
Speaker:Hence, any sample that shows a Q-Q plot like the one above can be well approximated by
Speaker:a normal distribution.
Speaker:Figure 6.
Speaker:Q-Q plots for NASDAQ-100 and APPL returns.
Speaker:The grid lines are at (-1.96, 0, 1.96) for both axes.
Speaker:Figure 6 presents the Q-Q plots for the NASDAQ-100 and APPL returns.
Speaker:The return from the NASDAQ-100 return shows a reasonable match with the normal quantiles,
Speaker:while the quantiles of the APPL return show substantial departures from the normal quantiles.
Speaker:This indicates that, while the NASDAQ-100 returns may be approximated by a normal distribution
Speaker:with reasonable accuracy, a normal distribution will be a poor approximation to the APPL return
Speaker:distribution.
Speaker:9.
Speaker:Concluding remarks.
Speaker:As an opening chapter, the basic concepts and descriptive measures of statistics were
Speaker:discussed with the following keywords -
Speaker:•Sample and population.
Speaker:•Mean and Median.
Speaker:•Standard deviation and Inter-quartile range.
Speaker:•Percentile or quartiles.
Speaker:•Histogram, Time plots, Q-Q plot, Box-Whisker plot.
Speaker:•Normal distribution.
Speaker:If you understand the listed concepts and methods, and you can apply them to real-world
Speaker:situations, you already have made big steps into the world of statistical thinking!
Speaker:You can produce these statistics using popular tools such as Excel.
Speaker:This has been the art of statistical thinking, detect misinformation, understand the world
Speaker:deeper
Speaker:and make better decisions. Advanced Thinking Skills Book 3. Written by Albert Rutherford,
Speaker:J. H. Kim, PhD. Narrated by Russell Newton. Copyright 2022 by Albert Rutherford. Production
Speaker:copyright by Albert Rutherford.
