Differential length and overlap with the stem in multiple-choice item options: A pilot experiment
Multiple-choice items are extensively used across different assessment contexts. A crucial requirement for ensuring their validity is their correct development, and a number of item-writing guidelines have been proposed that support item developers. This experimental pilot study aimed to investigate...
| Authors: | ; |
|---|---|
| Format: | Article |
| Publication date: | 2019 |
| Country: | España |
| Institution: | Universidad Autónoma de Madrid |
| Repository: | Biblos-e Archivo. Repositorio Institucional de la UAM |
| Language: | English |
| OAI Identifier: | oai:repositorio.uam.es:10486/689866 |
| Online Access: | http://hdl.handle.net/10486/689866 https://dx.doi.org/10.5093/psed2018a20 |
| Access Level: | Open access |
| Keyword: | Flawed items Item difficulty Item-writing guidelines Length of options Lexical overlap Multiple-choice items Psicología |
| _version_ | 1878734646483615744 |
|---|---|
| author | Casu, Giulia |
| author2 | García García, Carmen |
| author2_role | author |
| author_browse | Casu, Giulia García García, Carmen |
| author_facet | Casu, Giulia García García, Carmen |
| author_role | author |
| collection | Biblos-e Archivo. Repositorio Institucional de la UAM |
| description | Multiple-choice items are extensively used across different assessment contexts. A crucial requirement for ensuring their validity is their correct development, and a number of item-writing guidelines have been proposed that support item developers. This experimental pilot study aimed to investigate the effect of violating two item-writing guidelines: the differential length of the correct option compared to distractors and its lexical overlap with the stem. Standard and flawed items, respectively adhering to and deviating from guidelines, were randomly assigned to 55 college students and compared in their psychometric functioning. Results indicated that, in general, flawed items tended to be easier and less subject to random answers than standard ones, but significant differences were few. Discrepancies between standard and flawed subtests approached statistical significance with medium effect sizes. Although of interest, findings must be cautiously interpreted due to the small sample size. Implications for future research are discussed. |
| eu_rights_str_mv | openAccess |
| format | article |
| id | ES_6d6a2dcd3443f4eaf59a9d59bddf40da |
| institution | Universidad Autónoma de Madrid |
| instname_str | Universidad Autónoma de Madrid |
| language | eng |
| network_acronym_str | ES |
| network_name_str | España |
| oai_identifier_str | oai:repositorio.uam.es:10486/689866 |
| publishDate | 2019 |
| publishDateSort | 2019 |
| publisher | Colegio Oficial de Psicólogos de Madrid |
| publisherStr | Colegio Oficial de Psicólogos de Madrid |
| reponame_str | Biblos-e Archivo. Repositorio Institucional de la UAM |
| spelling | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment La longitud diferencial y el solapamiento con el enunciado en las opciones de ítems de opción múltiple: Un experimento piloto Casu, Giulia García García, Carmen Flawed items Item difficulty Item-writing guidelines Length of options Lexical overlap Multiple-choice items Psicología Multiple-choice items are extensively used across different assessment contexts. A crucial requirement for ensuring their validity is their correct development, and a number of item-writing guidelines have been proposed that support item developers. This experimental pilot study aimed to investigate the effect of violating two item-writing guidelines: the differential length of the correct option compared to distractors and its lexical overlap with the stem. Standard and flawed items, respectively adhering to and deviating from guidelines, were randomly assigned to 55 college students and compared in their psychometric functioning. Results indicated that, in general, flawed items tended to be easier and less subject to random answers than standard ones, but significant differences were few. Discrepancies between standard and flawed subtests approached statistical significance with medium effect sizes. Although of interest, findings must be cautiously interpreted due to the small sample size. Implications for future research are discussed. Los ítems de opción múltiple son ampliamente utilizados en contextos de evaluación muy variados. Un requisito muy importante para garantizar su validez es su correcta redacción, y para ayudar a conseguirlo se han desarrollado una serie de directrices. El objetivo de este estudio piloto experimental fue investigar el efecto del incumplimiento de dos de estas reglas, más concretamente, la longitud diferencial de la opción correcta comparada con los distractores y su solapamiento léxico con el enunciado. Para ello, se asignó aleatoriamente a 55 estudiantes a las condiciones de responder a ítems que respetaban o que incumplían las mencionadas directrices y se compararon las propiedades psicométricas conseguidas por los ítems. Los resultados indican que, en general, los ítems con incumplimientos tendían a ser más fáciles y a recibir menos respuestas aleatorias; no obstante, había pocas diferencias significativas y el tamaño del efecto era medio. Aunque de interés, estos resultados deben ser interpretados con cautela debido al escaso tamaño muestral. Se comentan las implicaciones para futuras investigaciones Colegio Oficial de Psicólogos de Madrid http://hdl.handle.net/10486/689866 https://dx.doi.org/10.5093/psed2018a20 |
| spellingShingle | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment Casu, Giulia Flawed items Item difficulty Item-writing guidelines Length of options Lexical overlap Multiple-choice items Psicología |
| title | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment |
| title_full | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment |
| title_fullStr | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment |
| title_full_unstemmed | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment |
| title_short | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment |
| title_sort | Differential length and overlap with the stem in multiple-choice item options: A pilot experiment |
| topic | Flawed items Item difficulty Item-writing guidelines Length of options Lexical overlap Multiple-choice items Psicología |
| topic_facet | Flawed items Item difficulty Item-writing guidelines Length of options Lexical overlap Multiple-choice items Psicología |
| url | http://hdl.handle.net/10486/689866 https://dx.doi.org/10.5093/psed2018a20 |