Math 151 , Day 33, Friday, April 19, 2002Hit reload to get current versionAfter class

Quiz returned:  If you got 7 or below you can repeat it (same quiz) for a max of 8 points, Monday after class or by appointment.

Significance testing: "an outcome that would "rarely" happen if a claim were true--is good evidence that the claim is NOT true."  Introduction Day31
Results of shoebox tests, and
A "Significance level" alpha is a probability level we decide on  in advance as being the "rarely" amount that will push us over
into believing (well, sort of) that the H₀ claim  is not true. (Historically older language than P-value)
We tend to use simple benchmark numbers for it, like .10 (1 in 10), .05 (1 in 20), .01 (1 in 100).
When the P-value is less  than (or equal to) a particular significance level alpha (say .05), we say,
    "The results are significant at the alpha = .05 level," or "The results are significant (P< .05)"
A particular scientific discipline may have a commonly accepted set of benchmarks, and language to go with it.
(I think I remember  .05 = "significant", .01 = "highly significant" in psychology?)
We will be less doctrinaire, use the language "significant at the alpha = ___ level."  (However, "nobody" uses a significance level
less rare  than .10, 1 in 10).

HW questions.

2-sided (2-tailed) test:
H₀: "Null hypothesis" A claim or statement about the population we would like to show is NOT true.
      H₀: µ =1000 hrs.  (Average lightbulb life.)
H_a: "Alternative hypothesis" A claim or statement about the population we are trying to find evidence FOR.  A value either much bigger than or much smaller than the H₀ value is evidence against H₀ & for H_a.
      H_a:   µ  Not = 1000 hrs. (Quality control on assembly line--find if it is "off" either way.)
   Sample of size n = 25.  Population sigma = 150 hrs.  Suppose xbar = 940 hrs. z = (940-1000) ÷ (150/5) = -2

 P-value: We measure the probability of seeing something (again) as extreme as the observed value (or more so).
So you need to measure the P-value symmetrically both directions from the observed value--so the P value is double what it would be for a one-sided test.  P-value is approximately 5%; more precisely, 2·.0228 = .0456
Our test is just barely significant at the .05 level; it is significant at the .06 level, the .10 level.  It's not significant at the .02 level or "higher".
Begin here Monday
Meaning of "significance"  (note--"High" significance means small alpha or P-value.)
Question: How do we know that .05 is "significant?" (.05 is 1 in 20 chance of seeing the result by "dumb luck" if the null hypothesis is true.)  Read sec. 6.3, pp. 343-345
>>Significance levels vary by field of study; different fields have different "customarily acceptable" levels.
      In reality, no sharp border between "significance" and "not significant"
>>How small a P is "convincing evidence" against H₀?  In practice...
        How plausible is H₀?  H_a?  Strong evidence needed to reject "conventional wisdom."
        How expensive (mentally, economically) will abandoning H₀ be?
>>"Statistically Significant" doesn't always mean "Important."  Big enough sample sizes will allow you to distinguish even small differences.
- - - - - - - - -
What if you don't have the Z-table but only have the t-table (Table C)?
What if you have a demanded level of significance, alpha?
    Table C gives a limited list of probabilities  across the top row: Right tail values for the bell distribution.
        The value in the bottom (z*) row under p is the corresponding standard normal value.
            "z* is the upper p critical value of the standard normal distribution."
  Do this: Find your z from the data. Make a sketch of the normal curve and mark z on it.  Mark the direction(s) of H_a.
    (If your z is in the direction of H_a , continue.  Otherwise the results are hopelessly not significant: you can quit.)
Find the z*'s in Table C that bracket your z (ignore minus sign).  Find the corresponding p's.
p      .02     .01
z*   2.054 \/ 2.326
       z = 2.111

So the P-value for your z is: between those 2 p's (one sided test)
                                           between double those 2 p's (two sided test)
    Test is significant at the bigger bracketing probability; not sig. at the smaller.
One sided: P-value is less than .02 and greater than .01
    Significant at the .02 level, not at the .01 level
Two sided: P-value is less than .04 and greater than .02
    Significant at the .04 level, not at the .02 level
If you have a specific demanded significance level, compare it with these levels.
            If  a test is significant at level b, then it is significant at every level bigger than b.
            If a test is Not significant at level d, then it is Not significant at every level smaller than d.
    "Significant at a":  probability of getting my results (again) by chance (if H₀ is true) is less than (or =) a.
     Significant at   Not significant at
p bigger  .10      .05      .01      .005     .001 smaller
                         /\
                        P-value
                        z-value (one-sided)
z* smaller 1.282   1.645_ | 2.326    2.576    3.091 bigger
  You can compare z directly to z* for your desired alpha. The 2-sided is a bit tricky.
          (2-sided: Split the alpha in 2, then find the z*.  Don't halve or double z's--it doesn't work!)

PreClass assignment Day 33 for  Day 34

Significance Tests. Activstats: Still in Ch 19, see previous days. Moore   6.3 next.  We'll skip 6.4.

HW Day33  ReRead (finish) 6.2 (pp337-8 optional), and read 6.3, especially pp. 343-345 for today.  Skip or skim Sec. 6.4. (Ch. 7 next.)
If you see "statistically significant" without a level, it often means "at the .05 level".
I suggest this for each problem that you find a P-value or sig. level for: sketch the curve representing the sampling distribution of x-bar, or of the z you calculate from x-bar, and mark your observational result on it (like fig. 6.10, 6.11, 6.13)

Hand in from Moore  Calculating p-value (one or two-sided), using z test statistic, relating to Sig. level  p. 333, 6.34 price reduc. on coffee   6.35 crankshafts true? Use your calculator to find the sample mean.   6.36 cola? Use your calculator to find the sample mean. = = = = = = = = = =  More p-values  p.341, 6.44 CEO pay = = = = = = = = =  Will be assigned Monday: Table C:  p.341, 6.48 CEO pay again (what you would do if you didn't have Table A) p. 341, 6.46, 6.49 general z statistic, significance,Turn the page--6.49 continues.  p. 342 6.50 patent protection; another z. = = = = = = = = = =  Fixed significance levels: if you only have table C, what can you say?  p. 337, 6.37 testing number generator 6.38 nicotine content = = = = = = = = = =  p. 342, 6.52 1% vs 5%    6.53 define stat. signif. p. 343, 6.54  knife edge .05 p. 345, 6.55 and 56 effect of n

Read,  to discuss

Optional  (more practice)  Calculating p-value (one or two-sided), using z test statistic, relating to Sig. level  p. 340 6.43 watered milk?