MATH 251, Probability and Statistics I, Fall 2005, Sept. 28, Day 15After class

Simpson's paradox:  An association or comparison that holds for all or several subgroups can reverse direction  when the data are combined into a single group.  Day 14
- - - - - - - - - - - - - - - - - - - - - - - -

Chapters 1 and 2 have covered analyzing data that was given to us--what it said about itself.  "Exploratory Data Analysis"

Ch. 3:  Producing Data:  Aim:  create data sets that will allow us to make inferences to a larger world than just the data we have.   Day 14
Design of experiments Sec 3.2    Special jargon!

Do something to:
    "Experimental Units" = "Subjects"   (cases)
Treatment:  Specific experimental condition.
Factor: Explanatory Variable which we manipulate.
    Levels: Specific values of a factor that we set.
Response variable(s)

E.g. 2 headache medications, in combination?
A two-factor experiment, each with 3 levels. 9 possible treatments.
    Factor A: Aspirin:  levels None, 500 mg, 1000 mg
    Factor B: Caffeine: levels None, 50 mg, 100 mg
Response variable: reported pain relief

			Aspirin
		None	500 mg	1000 mg
	None	Treatment 1	Treatment 2	Treatment 3
Caffeine	50 mg	Treatment 4	Treatment 5	Treatment 6
	100 mg	Treatment 7	Treatment 8	Treatment 9

Lurking and confounding variables:  Control--how?  Nothing except experimental treatment should differentially affect response.   Biased design: Systematically favors certain outcomes
    Compare responses under several treatments, look at differences.
Placebo effect: a positive response to a "sham" medical treatment--if you believe it will work, it very likely will. (Tinkerbell?)
    A medical treatment must be shown to be better than a placebo (at least) to be approved by the FDA.
               placebo="I shall please" (Latin)
To control for the placebo effect, All treatments should "look alike".  Treatment 1 above should be a pill with no medicine--a "placebo".  (Some experiments even try to duplicate side effects of actual medication.)

"Control group" Group that gets the "baseline"--"null"-- "none" or "placebo" level of the factor.  Should be "just like" the group(s) that get the "treatment" ("real" levels of the factor).  So Treatment 1 above will go to the "control group", the other 8 will go to "experimental" or "treatment groups."
Murky language here:  "Experimental vs. control" or "Treatment vs. control" is different usage from "Treatments", one of which is the "control"="none"/"placebo".
     *Sometimes the Control  is the current "best practice" treatment, rather than none.
Note:  sometimes in "hard sciences", no baseline; sometimes even no comparison.  Just Treatment-->Observe response.
Start here Friday
Comparative experiment:  How to get groups of experimental units  "just like" one another?  or at least not biased.
  Biased design: Systematically favors certain outcomes.
Randomize who goes into which group.  (Usually our batch of  experimental units is not a random sample from the population of all individuals-- usually volunteers, etc.)
Randomized comparative experiment : Diagrams of design, IPS pp. 202, 205
Completely randomized: all exp. units allocated at random among the treatments.

E.g. does acupuncture work for PMS?  Response: report of symptoms.
    One factor, 3 Levels:  None (music?), Acupuncture (wrong places), Acupuncture (right places). 3 treatments.
            (control(s)?)
      30 subjects with PMS:  Randomize, 10 each treatment.  Administer treatments.  Compare symptoms. (Do diagram)

Picking groups with Simple Random Sample applet www.whfreeman.com/ips :  Label your subjects with numbers 1 thru 30. Set "population" equal to 30.  Pick "sample" of size 10 from the 30; they get the first treatment. (Write down their numbers.)  From the remainder, pick another "sample" of size 10 for 2nd treatment.  (Keep track where it starts in the sample list.)   The 10 remaining get the 3rd treatment.  (Next time: using a paper Random Digits Table--pp. 203-4; sampling with SPSS)

Why 10 each, not just 1 each?  Repeat the experimental treatments on many units allows "averaging out" chance variation in the units.  (Don't confuse the repetition  needed within one experiment, with "replication" of the whole experiment in a different time and place to confirm its results.)

Principles of designing an experiment (p. 203) See above...

• Control effects of lurking and confounding variables, by comparing treatments, where levels of explanatory variables are controlled, and all other conditions are controlled to be the same.
• Randomize the assignment of units to treatments: reduces possibility of confounding characteristics of exp. units with the treatments.  "Control what you can, randomize the rest."
• Repeat the treatments on many exp. units to reduce the chance of unrepresentative results.

 

Designed experiments are our best shot at cause-->effect  but!
Need to "replicate" in different contexts, with variations.

Placebo and biasing effects can result from expectations of medical staff.  "Blind" means subject doesn't know who's getting "real" treatment.  "Double blind" means neither subject, nor staff administering treatment, nor people recording response variables know who's getting which treatment. (That should really be triple blind?) Most convincing for cause/effect.

Usually an experiment treats the placebo effect as a confounding variable, and is designed so placebo effect will work equally on all groups.  There is no attempt to measure the placebo effect.  ("All" drug studies.)
        PMS/acupuncture:  Acupuncture (wrong) vs. Acupuncture (right).
Sometimes an experiment deliberately tries to measure the placebo effect (as in the article).
        Acupuncture (wrong) vs. Music.

Lack of realism Do sociology, psychology, (medical?) experiments generalize to "real life?"  Time, money, subjects.  Ethical questions...