Experimental+Design+Handout-Basics

· Random Selection and Random Assignment (if we do a study where two groups are subjected to different tests) result in lower likelihood of results due to outside variables. __All of these aspects help improve the internal validity of the test:__ Internal Validity – results were caused by manipulation of the independent variable, NOT by other factors. Internal validity is basically the ability of a test to test for causation. “If ________, then__________.” “If not _________, then not _________.” Internal Validity can be increased by: -Controlling variables that aren’t being tested for (and possible confounding variables accounted for). Therefore, there is no other plausible explanation for a certain effect to be occurring other than the hypothesized cause. One-Group Design: Randomly select a group to go through both of the conditions repeatedly. Two-Group Design: Randomly assign the population into two groups: one of the groups is exposed to the manipulated variable, and the other group is the control group. Then the two groups are tested for any differences after the testing is done. Randomized Block Design: Do a Two-Group design, only block the population into sub-groups (i.e. separate athletes from non-athletes or split up girls and boys) before the subjects are randomly assigned to either the control or manipulated group. All hypothesis tests are conducted the same way. The researcher states a hypothesis to be tested, formulates an analysis plan, analyzes sample data according to the plan, and accepts or rejects the null hypothesis, based on results of the analysis. Test statistic = (Statistic - Parameter) / (Standard deviation of statistic) Test statistic = (Statistic - Parameter) / (Standard error of statistic) where //Parameter// is the value appearing in the null hypothesis, and //Statistic// is the [|point estimate] of //Parameter//. As part of the analysis, you may need to compute the standard deviation or standard error of the statistic. Previously, we presented common [|formulas for the standard deviation and standard error]. When the parameter in the null hypothesis involves categorical data, you may use a chi-square statistic as the test statistic. Instructions for computing a chi-square test statistic are presented in the lesson on the [|chi-square goodness of fit test]. Type I Error is a result of the null hypothesis being rejected when it is true. The probability of Type I error is the significance level, or alpha ( a ). Type II Error is a result of the null hypothesis not being rejected when it is false. The probability of not Type II error is the power of the test, or beta ( b ). Statistical Significance: significance level chosen at the beginning of the experiment. -probability of a true null hypothesis being rejected Test Power: probability of not making a Type II error (1- b ). Sufficient Testing Power considerations: How many subjects should be tested? What is the significance level chosen for the test?
 * HANDOUT - More info to come!**
 * __Aspects of Experimental Design to Consider:__**
 * Control** – a group in which the factor that is being investigated is not manipulated. This way, the results of the group in which the factor is manipulated can be compared to a control to make sure that the results are significant.
 * Randomization** – the groups need to be as homogeneous as possible, and randomization helps limit any bias that might occur due to any individual’s characteristics (so the treatment groups are as similar as possible)
 * Replication** (of experiment on large sample size) – improves the significance of the results of the experiments
 * Types of Experimental Design:**
 * __Four Steps to Experimental Design:__**
 * ** State the hypotheses ** . Every hypothesis test requires the analyst to state a [|null hypothesis]  and an  [|alternative hypothesis]  . The hypotheses are stated in such a way that they are mutually exclusive. That is, if one is true, the other must be false; and vice versa.
 * ** Formulate an analysis plan ** . The analysis plan describes how to use sample data to accept or reject the null hypothesis. It should specify the following elements.
 * Significance level. Often, researchers choose [|significance levels]  equal to 0.01, 0.05, or 0.10; but any value between 0 and 1 can be used. (Compare this to the p-value)
 * Test method. Typically, the test method involves a test statistic and a [|sampling distribution]  . Computed from sample data, the test statistic might be a mean score, proportion, difference between means, difference between proportions, z-score, t-score, chi-square, etc. Given a test statistic and its sampling distribution, a researcher can assess probabilities associated with the test statistic. If the test statistic probability is less than the significance level, the null hypothesis is rejected.
 * ** Analyze sample data. ** Using sample data, perform computations called for in the analysis plan.
 * Test statistic. When the null hypothesis involves a mean or proportion, use either of the following equations to compute the test statistic.
 * P-value. The P-value is the probability of observing a sample statistic as extreme as the test statistic, assuming the null hypotheis is true.
 * ** Interpret the results ** . If the sample findings are unlikely, given the null hypothesis, the researcher rejects the null hypothesis. Typically, this involves comparing the P-value to the [|significance level] , and rejecting the null hypothesis when the P-value is less than the significance level.
 * ||  |||| True Hypothesis ||
 * ||  || HO || H1 ||
 * Hypothesis chosen || HO || Okay || Type II Error ||
 * ^  || H1 || Type I Error || Okay ||
 * __Statistical Significance vs. Sufficient Testing Power__**