Advising on research methods: Selected topics 2013 results from a research master course Methodological Advice that was given at the University of Amsterdam at the end of 2013 by Gideon J. Mellenbergh and Herman J. Adèr.

The course had the same format as the course given in 2012.

The objectives of the course were: (a) to acquire methodological knowledge that is needed for advising researchers in the behavioral and social sciences, and (b) to get experience with methodological consultancy.
The main material for the course was the book:
 
Advising on research methods: A consultant’s companion
by Herman J. Adèr and Gideon J. Mellenbergh (with contributions by David J. Hand).
See: ARM book.

The students had to fulfill various assignments, one of which was to write a paper on a topic that may come up during methodological consultancy.

In the beginning of the course, paper topics were selected from a long list of relevant methodological issues.

The publication process of drafting, submitting, reviewing, adapting and correcting was the same as in the production of any other edited paper collection.

Six students participated in the course. They made the following contributions to the book:

Daan van Renswoude

compares random and nonrandom assignment of study participants to different conditions of a design. Random assignment spreads all participants’ characteristics randomly across conditions, whereas nonrandom assignment causes systematic differences between participants of different conditions. He illustrates the effects of non-random assignments on the bias of the estimates of condition means and on the Type I error of a test of the null hypothesis of equal condition means.

Joost Kruis

discusses the use of modern item response theory for the analysis of psychological and educational tests. He considers the one-parameter (Rasch-) and two-parameter (Birnbaum-) models for the analysis of test data. He illustrates the checking of model assumptions using simulated data. Moreover, he shows how the item information function can be used in item analysis.

Paul Lodder

introduces methodological consultants to modern methods for the handling of missing data. He discusses types of missingness and four methods to impute missing data. He illustrates these methods using the data of a 20-item depression questionnaire. Different types of missingness and different percentages of missing item responses were applied to the empirical questionnaire data, and their effects on the psychometric properties of the questionnaire were studied.

Abe Huijbers

introduces bootstrap methods. The assumptions of classical statistical methods are often violated in psychological and educational research. Bootstrap methods can be applied to construct confidence intervals and to test null hypotheses when assumptions of classical methods are violated. He discusses the well-known bootstrap method, and applies it to empirical data.

Lisa Wijsen

introduces data mining to methodological consultants. She describes similarities and dissimilarities of data mining and statistics. Moreover, she discusses the application of data mining in behavioral research. She illustrates data mining in educational research with categorical data of students who participated in arithmetic learning in the Math Garden (‘Rekentuin’).

Simon Columbus

discusses economic games and their application in psychology. He describes conceptual and methodological issues on the interpretation of economic games. Moreover, he compares economic games with other experimental designs that are used in psychological research.