From Norris & Ortega (2000, pp. 497-498). [bold emphasis is mine]
- Utilize simple designs that investigate only a few variables at most; interactions of variables should be investigated systematically across multiple experiments, not within single experiments.
- Incorporate pre-tests and post-tests as well as true control groups in experimental and quasi-experimental study designs, to identify better the amount of observed effects attributable to instructional treatments.
- Design studies with the replication of variables (not other studies) in mind; avoid the myth of the single decisive study by engaging in long-term research agendas in which a series of studies systematically provides data points about specific variables.
- Consider the validity of dependent variables in terms of the kinds of interpretations to be based on them; estimate and report the consistency or reliability of the use of outcome measures.
- Choose the analytic and interpretive techniques that will provide accurate answers to the research questions that are being asked; where used, interpret results of statistical significance tests appropriately.
- For questions about the presence of an effect, the size of an effect, or the importance of an effect, calculate effect sizes (statistical significance tests will not provide answers to any of these questions).
- Incorporate estimates of error (e.g., standard error, confidence intervals) into all quantitative analyses of experimental data.
- Report enough data about independent, dependent, and moderator variables such that related findings may be compared with other investigations of the same variables and such that future researchers will be able to replicate these variables; include observations about what actually occurred when variables were operationalized in investigations.
- Always report the data necessary to enable further interpretation and accumulation of study findings, including, but not limited to: means, standard deviations, and group sample sizes on all pre- and post-experimental measures (regardless of statistical significance); where used, report complete results of statistical significance tests (e.g., not just the probability levels or F or t values for findings that were observed to be statistically significant).
He aprovechado un descanso en mis estudios para compartir las recomendaciones de Norris & Ortega (2000, pp. 497-498) en el campo de la investigación cuantitativa en Adquisición de Segundas Lenguas. Por razones obvias, pese a ser más denso que el dulce de leche, es el artículo más citado de la revista Language Learning.
Soy algo escéptico con el uso de la significación estadística en la investigación del aprendizaje de segundas lenguas. El artículo en cuestión confirma mis sospechas: resulta complicado generalizar, sobre todo si no se tiene en cuenta el tamaño del efecto (effect size). En todo caso, entiendo la moda y la necesidad de los estudios cuantitativos.
De hecho, aunque el número de participantes para mis experimentos va a ser limitado (50 participantes, aproximadamente), voy a combinar métodos cualitativos y cuantitativos, teniendo en cuenta las recomendaciones y los ejemplos que proponen Norris & Ortega para los últimos. Ni que decir tiene que intentaré seguir un rigor paralelo, si no mayor, en el estudio cualitativo.
Por cierto, uno de los muchos “bonus” del artículo es una jerarquía de efectividad en la instrucción de segundas lenguas:
Focus on Form explicit > Focus on Forms explicit > Focus on Form implicit > Focus on Forms implicit