A graphical user interface for dexter

Add scoring rules

After starting a new project you have to import a scoring rules file. This is because dexter needs to know the scores for every permissible response for each item.

The scoring rules file is an excel or csv file and can have on of two formats

• columns: item_id, response, item_score, listing all items and possible responses.
• columns: item_id, nOptions and key. Keys can be either alphabetical or numeric.

In both cases the first row must contain the exact column names. Below you see examples of the two formats.

If you have open ended items, these can be accommodated by scoring them before importing. The column response should in that case be equal to the column item_score (i.e. response 0 gets score 0, response 3 gets score 3 and so on).

After selecting a file, dexter will show a preview of the rules. If all looks ok, you should press Import to import the scoring rules.

Import response data

Once you have imported the rules, the next step is to import response data. To import response data, select the Data import tab in the navigation bar on top of the page.

Response data is imported one booklet at a time. First provide a unique name for the booklet or testform in the field booklet id. Next use the Browse button to select a file with response data. This file can be in excel or csv format and the first row must contain column names.

The column names should be item id’s which each have an exact match to the item id’s in your scoring rules, note that capitalization matters. You can also include person properties like grade or study program. It is also good practice (but not mandatory) to include a column person_id with a unique identification of each person in your data. Each row in your data, apart from the column names, should represent a single person like in the example below.

When you select a file, dexter shows a preview of the response data. Any column with a name that doesn’t match a known item id will be ignored by default. You can import such columns as person properties by clicking the corresponding buttons.

If the preview looks all right you can import the response data by pressing the import button.

Booklets

The booklets tab shows a table of classical statistics per booklet and plots of the item total regressions.

The plot shows regressions based on two models: the interaction model is shown with a thicker but lighter line, and the Extended Rasch model is shown with a thinner, darker line. The unsmoothed data is shown as red dots. The curtains are drawn at the 5% lowest and the 5% highest sum scores. This can all be adjusted with the input fields on top of the plot.

The Rasch model fits this item very well, so the two curves practically coincide. This need not be always the case. The interaction model is quite interesting for practice. It shares the conditioning properties of the Rasch model, which means that we can predict the expected item score at each observed total score. Hence, we have the possibility to represent the model and the observed data on the same plot, without tricks or circular reasoning. This makes the interaction model a very useful tool to evaluate the fit of the Rasch or Extended Nominal.

Items

The items tab shows classical statistics for the items on the left, including RiT, RiR and pvalue.

The distractor plot on the right shows a non-parametric regression for each response on the test score. In this case a multiple choice item which obviously has poor distractors.

Note: if you spot a key error, the table of responses right under the distractor plot allows you to change the score for a response by clicking on the respective score and changing it.

Abilities & Plausible Values

When you have calibrated your model dexter can compute ability estimates for each person. You have a choice of using a Maximum Likelihood Estimate (MLE) or an Expected a Priori (EAP) estimate with choice of a Jeffreys or a normal prior.

Plausible values are random samples from the posterior distribution of ability. They are closer to the true distribution of ability in the population than straightforward ability estimates. Plausible values are continuous, as opposed to ability estimates which are granular because of the discrete nature of test scores. Therefore the distribution is much smoother.

There is an extremely rich selection of plots to see how ability is distributed in your population(s). Enriching your data with person properties (see the projects tab) will let you make the most of these. You can also view or download the estimated abilities per person.

Score-ability tables

Score-ability tables lets you compute a score-to-ability transformation table for each booklet. This is often desirable, e.g. for automatic systems or score transformation tables supplied with paper tests for scoring by teachers.

Items

After you have fitted a model, the item tabs shows the estimated parameters and a graphical item fit measure.