Math 151 , Day 38, Monday, Dec. 1, 2008 .After class, links fixed. Hit reload .

Exams back. Discussion done today.  Solutions.
Wed. Dec. 17, 9-12 a.m.  If this is a problem for you, please email me very soon.  
   Alternative--Monday afternoon (The favorite so far.), Tuesday morning/afternoon?? (Email your possibilities; I'll pick one!)
  Full exam schedule is at   http://www.wells.edu/pdfs/finals.pdf
     Registrar's page with link to this and other good stuff: http://www.wells.edu/academic/regist.htm
Ch. 15: "Significance tests use an elaborate vocabulary, but the basic idea is simple: an outcome that would "rarely" happen if a claim were true--is good evidence that the claim is NOT true." (p.363 top)
Day 35 for other details.  Summary, comments:

We went through the machinery of testing and used the Applet:  P-value of a test of significance. Will finish the work on this page next time.

Take data.  Calculate test statistic. For µ, test statistic is the z-score of xbar. (Start with xbar, standardize using mean of H₀)
    Is it an unlikely result if  H₀ is true?  Then that is evidence against H₀.
HW questions?  Day 37: 15.8, 9, 10

Measuring the strength of the evidence against H₀ (a common measuring stick for all distributions and parameters)--how weird is my observation if H₀ is true?:
P-value of a test:  The probability, computed assuming that H₀ is true, that the observed outcome would take a value as extreme or more extreme than that actually observed (if we could repeat taking-data again).  p. 368. Table A
    The smaller the P-value, the stronger the data's evidence against H₀ ( for H_a).

For a test of µ  , using xbar (sigma known), the P-value is
--the area of the tail beyond the observed xbar, in the direction of H_a (one-sided)
(--or twice that area (two-sided).)
<>Applet:  P-value of a test of significance automates this.  (Uses "raw" scale of xbars, rather than z-scores).  Table A
HW questions?  Day 37: 15.12, 13, 14 (one-sided).  11 (two-sided.)
Other HW questions on machinery? 18, 19, 37(2sided), 38(?2sided), 42, 43(2sided), 44(wrongP)

As n increases, the P-value corresponding to a particular x-bar (if it's in the correct tail) gets smaller. Applet:  15.40
    Makes sense: more data; more evidence against H₀ (if it's actually not true)
      Same issue as: Effect of sample size on distribution of x-bars: NormalandXbar.xls

A "Significance level" alpha is a probability level we decide on  in advance as being the "rarely" amount that will push us over into believing (well, sort of) that the H₀ claim  is not true. (Historically older language than P-value.  Appropriate levels vary by discipline.)
We tend to use simple benchmark numbers for it, like .10 (1 in 10), .05 (1 in 20), .01 (1 in 100).
When the P-value is less  than (or equal to) a particular significance level alpha (say .05), we say,
    "The results are significant at the alpha = .05 level," or "The results are significant (P< .05)" .  Giving actual P is better, if you can.
 Lightbulbs:  One-sided:  .0228 = P-value.  More than  2% and less than 3% chance of getting a result this far out (in this direction) if we did it again.
          "Significant at the alpha =.03 level.  Also at the alpha = .05 level"  (P-value says,  rarer than these levels)
          "Not significant at the alpha = .02 level.  Also not significant at the alpha = .01 level"  (P-value says, more common than these levels)
   Two-sided:  .0456 = P-value.  (Barely) less than 5% chance of getting a result this far out if we did it again.
            "Significant at the alpha = .05 level. (Also at alpha = .10).   Not significant at the alpha = .04 level.  Nor .01 level.
Applet:  Statistical Significance
You can pick the alpha you desire, and see if your x-bar lies outside the "alpha" barrier(s). (approach of p. 376-79) But P-value is more informative.
HW 15, 16, 17

"Real" shoebox data.  Last term
This term Updated.  (I can see that several yellow samples with mean 25 or above haden't been plotted yet.)  Are now (Wed.)

- - - - - - - - - - - - - - - -
What if you don't have the Z-table but only have the t-table (Table C)?
What if you have a demanded level of significance, alpha?
    Table C: a limited list of probabilities  across the bottom rows:
            = Tail values for the bell curve distribution.   (one sided = one tail, two sided = two symmetrical tails)
        The value in the z* row above  P is the corresponding standard normal value ("critical value"). 
                 Check z* = 1.960,  prob. .025 above it (or below -1.960).  .05 farther out than it.  Corresponds to Table A.
      
  Do this: Find your z from the data. Make a sketch of the standard normal curve and mark your z on it.  Mark the direction(s) of H_a.
    (If your z is in the direction(s) of H_a, continue.  Otherwise the results are hopelessly not significant: you can quit.)
Find the two z*'s in Table C that bracket your z (ignore minus sign).  Find the corresponding P's.
    e.g. z =2.111
                                                 z = 2.111
      z*         2.054 \/ 2.326
One-sided P  ...  .02     .01
Two-sided P  ...  .04     .02 

      So the P-value for your z is: between .02 and .01 (If it's a one sided test)
         &  between double those 2 p's--between .04 and .02 (If it's a two sided test)
    Test is significant at the bigger bracketing probability; not sig. at the smaller.
One sided: P-value is less than .02 and greater than .01
        Significant at the .02 level,not at the .01 level
Two sided: P-value is less than .04 and greater than .02
        Significant at the .04 level,not at the .02 level
If you have a specific demanded significance level, compare it with these levels.
            If  a test is significant at level b, then it is significant at every level bigger than b.
            If a test is Not significant at level d, then it is Not significant at every level smaller than d.
    "Significant at a":  probability of getting my results (again) by chance (if H₀ is true) is less than (or =) a. My result is less common than a.
Results   Significant at    Not significant at
p bigger  .10      .05      .01      .005     .001 smaller
                        /\
                        P-value (one-sided)
                        z-value 2.054
z* smaller 1.282   1.645  | 2.326    2.576    3.091 bigger
  You can compare z directly to z* for your desired alpha.  z >z*?  Significant at that alpha.  
    The 2-sided is a bit tricky.  Don't halve or double z's, ever!--it doesn't work!)

Ch. 15: "Significance tests use an elaborate vocabulary, but the basic idea is simple: an outcome that would "rarely" happen if a claim were true--is good evidence that the claim is NOT true." (p.363 top)
I'm not making it up that this idea is important:  Financial Times (influential and high-end British newspaper) this winter:
   (with formatting and pictures)  (without)   Statistical Significance: #10 of "The Ten Things Everyone Should Know About Science"

- - - - - - - More NEW STUFF- -...  -  - -
back to Ch. 16, cautions:  (Same old: SRS, normal pop. or Xbar, sigma known)
>>How small a P is convincing evidence against H₀?  (What alpha, to "Reject H₀?)
   --Is H_a surprising?  (Entrenched opinion is "for" H₀ .  )  Need strong evidence (small P).
   --Is rejecting  H₀ expensive?  Need strong evidence for H_a. 
        [May need to repeat experiment for doubters]
No sharp border between "significant" and "not"--though decisions may need to be made.

>>Statistical significance is not the same as practical significance ("clinical significance") 
      Tiny difference (from null value) can be statistically significant if sample size is large.
       Big difference may not be statistically significant if sample size is too small.
Do confidence intervals:  Estimate the size of the effect, not just yes/no of test..

More cautions next.