Additional Information about Methodology and Analysis
Multi-Institutional Research: Examining Change over Time
A univariate analysis confirmed that cross-institutional comparisons would not give meaningful results for individual paper sets (i.e., P1, P2, P3), because the universities’ student populations were so different, as manifested in the differences in P2 scores, F(3,109) = 7.30, p < .001. When the change in total mean scores was calculated from P1 to P2, and from P2 to P3, however, the differences in those changes from university to university were non-significant (i.e., P1 to P2: F(2,54) = 1.29, ns). These results suggest that our multi-institutional data set, taken as a whole, can be analyzed to assess change over time.
Description of Statistical Analyses Used
Regression analysis was chosen to evaluate the effect of variables on the change in writing scores from P1 to P2 and from P2 to P3. (The assumptions of linearity, normally distributed errors, collinearity, and uncorrelated errors were checked and met.) Skewness value for SourcesApplied and PriorKnowledgeMinusChallenge are slightly above the recommended maximum of 1.96 (2.19 and 2.20, respectively).
For both writing-change-of-score variables, potential independent variables were chosen by examining Pearson correlations and entered into the regression model in stepwise fashion after controlling for the University factor. Despite the low number of cases for the P2-P3 analysis, the G*Power calculator (Faul, Erdfelder, Lang, & Buchner, 2007) indicates that given the effect size the power was sufficient (1 – β > .80). Moreover, when the data were evaluated using a repeated measures ANOVA, we obtained substantially the same results, thus lending greater confidence to our results (Warner, 927).
Linear Regression Analysis Predicting P3 Minus P2 Mean Scores (N = 35)
|Predictor Variable||Β||SEB||β||R2||Adj. R2||∆ R2||∆ F|
|Total Sample (n = 35)|
|Sources Applied||.16||.06||.36*||.25||. 21||.17||8.07**|
|All Prior Knowledge||-.27
|Metacognition: Indiv Descrip||.13||.05||.44||.48||.42||.10||6.26*|
*p < .05
**p < .01
We note that while the findings are statistically significant, they are not robust because the sample size is rather small (58 for P1/P2, 35 for P2/P3). Larger sample sizes in future studies will provide greater confidence that these promising findings are generalizable across varying populations.
Writing and the Question of Transfer: Writing about Writing
Dana Driscoll, Gwen Gorzelsky, and Ed Jones, participants in the Elon Research Seminar on Critical Transitions: Writing and the Question of Transfer, discuss their multi-institutional research on Writing about Writing as a pedagogy that facilitates writing transfer.