Math 151 , Day 32, Wednesday, April 18, 2001 final ver (see HW)

Significance testing, continued:
Question: How do we know that .05 is "significant?" (.05 is 1 in 20 chance of seeing the result by "dumb luck" if the null hypothesis is true.)  Read sec. 6.3, pp. 343-345
>>Significance levels vary by field of study; different fields have different "customarily acceptable" levels.  In reality, no sharp border between "significance" and "not significant"
>>How small a P is "convincing evidence" against H₀?  In practice...
        How plausible is H₀?  H_a?  Strong evidence needed to reject "conventional wisdom"
        How expensive (mentally, economically) will abandoning H₀ be?
>>"Statistically Significant" doesn't always mean "Important."  Big enough sample sizes will allow you to distinguish even small differences.

Questions on last HW?

What if you don't have the Z-table but only have the t-table (Table C)?
What if you have a demanded level of significance, alpha?
    Table C gives a limited list of probabilities  across the top row: Right tail values for the bell distribution.
        The value in the bottom (z*) row under p is the corresponding standard normal value.
            "z* is the upper p critical value of the standard normal distribution."
  Do this: Find your z from the data. Make a sketch of the normal curve and mark z on it.  Mark the direction(s) of H_a.
    (If your z is in the direction of H_a , continue.  Otherwise the results are hopelessly not significant: quit.)
Find the z's in Table C that bracket your z (ignore minus sign).  Find the corresponding p's.
     z = 2.111
z*   2.054 /\ 2.326
p      .02     .01
So the P-value for your z is: between those 2 p's (one sided test)
                                           between double those 2 p's (two sided test)
    Test is significant at the bigger bracketing probability; not sig. at the smaller.
One sided: P-value is less than .02 and greater than .01
    Significant at the .02 level, not at the .01 level
Two sided: P-value is less than .04 and greater than .02
    Significant at the .04 level, not at the .02 level
If you have a specific demanded significance level, compare it with these levels.
            If  a test is significant at level b, then it is significant at every level bigger than b.
            If a test is Not significant at level d, then it is Not significant at every level smaller than d.
    "Significant at a":  probability of getting my results (again) by chance (if H₀ is true) is less than (or =) a.
          Not significant at      Significant at
p smaller  .001     .005     .01     .05     .10   bigger
                                 /\
                              P-value
                              z-value (one-sided)
z* bigger 3.091    2.576    2.326 | 1.645   1.282   smaller
  You can compare z directly to z* for your desired alpha. The 2-sided is a bit tricky. (2-sided: Split the alpha in 2, then find the z*)