Part I: Construct Development and Scale Creation
1) Choose a construct you would like to measure.
The construct will measure the functionality of working memory (WM) at the workplace, with the example of the performance of bookkeeping tasks.
2) Create an operational definition of your construct using at least three peer reviewed journal articles as sources as references.
Since individual’s WM capacities differ not only among people but also between different tasks or settings (Salthouse & Babcock, 1991), measurements should relate to the field of transfer, in this case – performance bookkeeping tasks. Such tasks typically require transfer of data from one medium to another, e.g. from a list to a computer (Carruthers & Espeland, 1991), and WM is typically defined as the quantity of information recalled 30 seconds after the learning phase (Baddeley, 2003). Hence, the operational definition in this domain should focus on the relative amount of pairs of financial data (i.e. name of account and a sum) recalled out of 15 pairs.
3) Select and list five items used to sample the domain.
- Work experience in this position.
- Educational background
- Lateral orientation (i.e. right- or left-handed)
4) Select method of scaling appropriate for the domain. Justify why you selected the scaling method you did.
The threshold for good WM functionality is defined as an active keeping of roughly, 7 ± 2 items for a few seconds (Smith, 1991). In this domain, an item is worthless unless the name and its corresponding figure are recalled; hence, as “item” is referred to as a pair. Therefore, the scale will have six levels, while a recall of 7 pairs is the lowest score for the upper half (median=6). 2 items will be used as a standard deviation. As a result, the scale will be:
1 – 0-1 item (3rd SD = exceptional impairment)
2 – at least 3 items (2nd SD = abnormal impairment)
3 – at least 5 items (1st negative SD = mild impairment)
4 – at least 7 items (1st positive SD = good performance)
5 – at least 9 items (2nd SD = excellent performance)
6 – at least (3rd SD = exceptionally good performance)
5) Format the items into an instrument with which you would query respondents. Justify whether this is an interview or self-report instrument.
The instrument is basically a means of objective measurement. The respondent will receive a list of 15 pairs of “transactions,” each one has a name of account and a negative (denoted in brackets) or positive sum (e.g. cash: ($20); payables: $57). The names will be composed from one word and the sums will not exceed 99. The respondent will then receive a learning phase of two minutes.
After the learning phase there will be a 30 seconds pause. Then the respondent will be asked to type all the “transaction” she can recall within a timeframe of 4 minutes. One point will be given to each pair (point=item, as discussed above).
Part II: Analysis and Justification
Working memory (WM) is a key cognitive skill for work and life (Baddeley, 2003). At the workplace, impairments in WM typically underpin slow, inaccurate and inconsistent task performance. The instrument proposed above may be used to several purposes:
First, employers can use it to screen candidates for bookkeeping positions. Second, it can help to investigate whether one’s weak performance is due to impairments in working memory or any other reason. Finally, if WM enhancement training program is administered, the instrument can be used to monitor changes in individual’s WM skills in a much more accurate way than actual job performance.
There are, however, several limitations to the instrument, which deny at the moment the validity of its performance standards (as discussed above). In order to validate the instrument, at least 100 representatives from each sampling category should be tested, and their results will serve for standardization for each category and the respective sub-categories (e.g. 25-30 years of age + female + 3-5 years experience, high school graduation + right handed). Moreover, it will be needed to test the external validity of the instrument by comparing test scores to the actual productivity of the respondents. That is, the query will check whether higher (or lower) test scores correspond with productivity, compared to other respondents with higher/lower scores from the same demographic groups.
After the standardization and validation processes, the results can be generalized to bookkeepers, whose performance is based on carrying on simple, repetitive posting tasks (Carruthers, & Espeland, 1991).
It is also possible to examine the reliability of the instrument by administering at least one more objective WM measurement, which has empirical findings to support its validity, to all the respondents, and then to compare the results of the two tests. Obviously, since the instrument use different scales, such a comparison will be based on relative comparisons against the other respondents in the reliability study. Like the instrument, the other WM should be also standardized as of demographic factors and its stimuli should be similarly simple.
Another issue that concerns the instrument’s validity and reliability is the settings and the items used during the test. The settings must be equal to all respondents, with emphasis on those factors that may influence attentional performance. Such factors include, among others, time of the day, the graphical appearance of targets and distractions such as noise or attractive visual stimuli in the room.
The items (i.e. the targets or elements) selected for the instruments play a key role in the validity of the measurement. Validity can be increased by ensuring fidelity of the items to the actual elements at the workplace (which is the field of transfer). Therefore, it is better to use items that look as similar as possible to the work materials, and the computer input system (target) will ideally be similar to accounting software. This is important because it is possible that internal factors of the instrument and/or the real work environment create certain bias on one’s performance.
Finally, WM skills should be isolated from other cognitive and/or professional skills. This implies that there should not be any additional challenges except the memorizing task, or else the extra “computing power” demanded from the respondents will most probably harm the results. In addition to the factors mentioned earlier (e.g. distractions and other attentional issues), the design should reduce to minimum difficulties such as abstract targets, interferences during the 30 seconds break, irregular text sizes and so on. In short, the test must be kept as simple as possible, while its power will be in the internal validity to the specific WM skills in question and the ability to externalize it to the whole industry.
- Baddeley, A. (2003). Working memory: Looking back and looking forward. Nature Review Neuroscience, 4(October), pp. 829-839. Retrieved August 2, 2009 from http://www.csuchico.edu/~nschwartz/baddeley.recent.pdf
- Carruthers, B. G., & Espeland, W. N. (1991). Accounting for rationality: Double-entry bookkeeping and the rhetoric of economic rationality. The American Journal of Sociology, 97(1), pp. 31-69. Retrieved August 2, 2009 from http://links.jstor.org/sici?sici=0002-9602%28199107%2997%3A1%3C31%3AAFRDBA%3E2.0.CO%3B2-%23
- Salthouse, T. A. & Babcock, R. L. (1991). Decomposing Adult Age Differences in Working Memory. Developmental Psychology, 27(5), pp. 763-776. Retrieved August 2, 2009 from http://22.214.171.124/scholar?q=cache:KHywaRjR7H0J:scholar.google.com/+working+memory+construct&hl=en
- Smith, E. E. (1991). Working memory. In Wilson, R. A. & Keil, F. C., The MIT Encyclopedia of the Cognitive Sciences, Cambridge, MA: MIT Press, pp. 888-889.