Interpretation and Evaluation of Research
*This course is required, but does not count toward the Ph.D. degree

This course will improve a students' understanding of the empirical research literature in education and psychology. It concentrates on developing the conceptual foundations of empirical research and the practical analytic skills needed by a competent reader and user of research articles. Topics address purpose statements, hypotheses, sampling techniques, sample sizes and power, instrument development, internal and external validity, and typical quantitative research designs. Exercises emphasize the critical evaluation of published research. 

Critical Perspectives on Research Methodology

This course addresses the major issues of educational assessment, with emphasis on the characteristics, administration, scoring, and interpretation of both formal and informal assessments, including but not limited to tests of achievement. All forms of assessment are examined including observation, portfolios, performance tasks, and paper-and-pencil tests, including standardized tests. Basic techniques of test construction, item writing, and analysis are included. Standardized norm-referenced tests and statewide testing programs are also examined.

Instrument Design and Development

This course is designed to familiarize students with the strategies, techniques, tactics, and issues in the development and administration of survey instruments. It will emphasize theoretical, measurement and practical considerations in the development of attitudinal instruments. The development and analysis of data resulting from several types of measurement scales will be covered.

Psychometric Theory I: Classical Test Theory and Rasch Models

Presents a study of theoretical concepts, statistical models, and practical applications in educational and psychological measurement. General topics include the history of measurement, Thurstone and Guttman scales, classical true-score theory, and item response theory. Specific topics include principles of Rasch measurement, parameter estimation procedures, fit statistics, item banking, and computer adaptive testing

Intermediate Statistics 

Topics and computer exercises address tests of means, partial and part correlations, multiple regression, analysis of variance with planned and post hoc comparisons, analysis of covariance, repeated measures analysis, elements of experimental design, and power analysis.

Evaluation Practice and Methods

This course introduces the process of conducting evaluations from beginning to end. Evaluation is a form of applied social science research focused on systematically assessing the value--merit, worth, or significance--of interventions. Taking an interdisciplinary approach to evaluation, this course draws on scholarly articles and examples from multiple fields including education, public health, social services, and international development and addressing evaluation at various scales including program, organizational, and systems-level evaluations. By the end of the course, students will gain knowledge of how to carry out evaluations; apply this knowledge to develop evaluation plans for real-world interventions; and gain skills to critique existing evaluations supporting their development as informed, critical consumers of evaluations.

General Linear Models

Addresses the construction, interpretation, and application of linear statistical models. Specifically, lectures and computer exercises cover ordinary least squares regression models; matrix algebra operations; parameter estimation techniques; missing data options; power transformations; exploratory versus confirmatory model building; linear-model diagnostics, sources of multicollinearity; diagnostic residual analysis techniques; variance partitioning procedures; dummy, effect, and orthogonal coding procedures; and an introduction to structural equation modeling.

Multivariate Statistical Analysis

This course provides lectures, examples, and lab analyses that address multinomial and ordinal logistic regression models, multiple group discriminant analysis, cluster analysis, multivariate analysis of variance, principal component analysis, factor analysis, and structural equation modeling. We cover various issues related to research design, model building, and the interpretation of the output from SPSS, R, Lisrel, and SAS software programs.

 Design of Experiments & Casual Inference

This course will introduce experimental design as a paradigm for thinking about the conduct of educational research and evaluation. The goals of this course are to introduce students to the design and statistical principles of the experimental approach to educational research with particular emphasis on the correct analysis of data arising from designed experiments. We will discuss a variety of experimental designs, their advantages and disadvantages, estimation of treatment effects, and significance testing. The topics covered will include the underlying logic of experimental and quasi-experimental designs, regression discontinuity and factorial designs as well as cluster randomized and multi-site trials.

Dissertation Seminar

This two-semester seminar is designed to assist doctoral candidates in the preparation of a formal doctoral dissertation proposal. All aspects of dissertation development will be discussed (e.g., problem development, human subjects review, final defense). Students will develop and present a series of draft proposals for faculty and student reaction. Depending on the circumstances of the student, an acceptable pre-proposal (Intent) or full dissertation proposal is required for completion of the course.

Proseminar

Non-credit, required of all first-, second-, and third-year students.

Design of Qualitative Research

 Introduces the foundations and techniques of carrying out qualitative research. Topics include philosophical underpinnings, planning for a qualitative research project, negotiating entry, ethics of conducting research, data collection and analysis, and writing/presenting qualitative research. Requires a research project involving participant observation and/or interviewing.

Evaluation Theory and Research

Evaluation as a professional practice and academic discipline is characterized by variation in perspectives on what evaluation is and how it should be practiced. This course introduces students to evaluation theory, key debates, and research on evaluation in order to develop students as critical, responsible evaluators and evaluation scholars.

Mixed Methods Research 

Course desciption coming soon.

Assessment Programs

Computers are widely available in schools and are increasingly used for large-scale testing programs. This course examines cutting-edge applications of computer-based technologies to the technology of testing and assessment. Among the topics explored are validity issues specific to computer-based testing; accessibility, universal design, and computer-based testing; computer adaptive testing; simulation-based and multimedia tests; and computer scoring of writing. The course encourages students to explore ways in which computer-based technologies can be used to enhance assessment and solve challenges to current approaches to student assessment.

 Advanced Topics in Evaluation

This course introduces students to theory, practice, and methods for evaluating systemic change. Whereas conventional evaluation draws on social science methods to evaluate discrete policies and programs, systemic approaches to evaluation draw on theory and methods from the systems and complexity sciences to support change or transformation in the face of complex issues and large-scale systems. Focal topics covered include differing rationales for and resistance to systemic change; ways to bring diverse actors and perspectives together; five phases of a systemic change process (i.e., shared vision, systems mapping, leverage points, navigating futures, and learning and adapting); and ways to evaluate the value and impact of a systemic change effort. Course materials span multiple fields (e.g., health, international development, sustainability, education), bridge academic scholarship and practical guidance, and incorporate popular media. By the end of the course, students will develop knowledge of systems and evaluation concepts and methods and apply new skills and experiences to create an original proposal to design and evaluate a systemic change initiative. Additionally, the interdisciplinary and hands-on classroom environment and peer review exercises provide opportunities to build community and shared learning.

This course is useful for students seeking careers in research/evaluation or leadership/management whether in government, philanthropy, non-profits, or international organizations. Prerequisites include one or more courses in (a) research or evaluation methodology (b) an applied field or problem area within a field (e.g., educational inequality, restorative justice, elder care); and (c) professional or personal experience with societal systems in need of change (e.g., activism, leadership, teaching).

Seminar in Evaluation and Public Policy

The course will examine some issues that arise in educational accountability. The purpose is to develop a deeper understanding of the policy issues and a critical appreciation of the relevant methodological strategies. Topics include the establishment and use of state-specific performance standards under NCLB and the quantification of the relative rigor of those standards; the evaluation of school or teacher effectiveness using so-called value-added models; and the policy evaluation of school reform efforts (such as charter schools) using data from large-scale cross-sectional surveys. The latter two topics both involve causal inferences from observational studies.