Q. How are articles coded?
A. A coding worksheet similar to a questionnaire was designed to answer basic questions about the content and context of each individual study. These questions were divided into 5 coding categories (Interventions, Subject/Content Area, Study Characteristics, Evaluations, Outcomes).
The structure and content of the coding worksheet evolved over a period of months to reflect the nature of the literature (see article screening and selection). After the original set of approximately 150 articles had been coded, all the open-ended entries under "other" for each category were listed and new items were developed to capture them. One category, the "Purpose" of the study, was dropped because too few articles stated their purpose, aims, or goals, and doing justice to the stated purposes would involve entering them verbatim into the database. All of the above-mentioned changes were entered onto an updated coding worksheet. A draft of the updated worksheet organized to reflect the proposed structure of the database was sent to select leaders in the field of engineering education for review and comment. Revisions to the worksheet were made as necessary. Then, all of the articles that had been accepted into the database were re-reviewed and re-coded onto a revised worksheet.
Some of the items in some of the categories on the worksheet and in the database are not mutually exclusive. This reflects our decision to capture the studies as described and be as user-friendly as possible (i.e., represent how the field thinks about the categories) rather than to force the studies into rigid categories. For instance, under the category Intervention and sub-category Teaching and learning, we list both Active Learning and a number of active learning strategies because some articles used an active learning strategy without stating a specific strategy. Further, the items in the worksheet and our coding reflect statements made in the article, not our interpretations of them. For instance, if an article said that students worked in teams, then, in the category Study Characteristics and the sub-category Social arrangement, we coded the item "Team", which connotes a division of roles and responsibilities, when, in fact, the social arrangement may really have been loosely formed small groups.