Table Of Contents

- Manual
- Getting Started
- Starting the Program
- Retrieving Data
- Manipulating Data
- The Variable List
- The Variable List Menu
- Filter Observations/Selecting
- Add New Variables
- Delete Variables
- Edit Metadata
- Set Replicate Weights
- New Variable Reserve
- Edit Value Labels
- Dummy Code Categorical Variable
- Collapse Categories of Categorical Variable
- Set Missing Values
- The Expression Evaluator

- Saving and Re-running Actions

- Sampling
- Procedures
- Measurement Models
- MML Models for Test Data
- Other Available Procedures

- Graphics
- Tools
- Estimation Methods
- Optimization Techniques
- Variance Estimation

- Post-hoc Procedures
- More user input instructions
- The User Interface
- Input Instructions
- Options
- Output Precision

- Glossary of Terms and Symbols

- Getting Started

BIB Spiraling

The technique known as Balanced Incomplete Block (BIB) spiraling, involves dividing the total number of assessment items into small units, called blocks, where each block is designed to take a short amount of time to complete. Each of these blocks is assigned to a number of distinct assessment booklets containing a small number of blocks in such a way that each item block is paired with every other item block in at least one of the booklets. This is the balanced part of the method. The total set of booklets is then spiraled, that is sorted into a systematically ordered sequence, and given to students so that every item block and every pair of item blocks is administered to a representative sample of students. Under this design, no student responds to more than a fraction of the complete pool of items, resulting in a substantial fraction of missing data for each student. This is the incomplete part of the method. BIB spiraling is particularly important in large scale assessment because it permits the administration of short individualized tests while retaining wide subject coverage.

The purpose of large scale assessments is typically to estimate the performance in particular subject areas of various subgroups of students rather than of individual students themselves. To accomplish this purpose, there is no need for precise measures for any individual student. Lord (1962) and Sirotnik & Wellington (1977) have demonstrated that population characteristics can be estimated accurately without first obtaining accurate estimates for individuals. Hence, it is not necessary or even desirable on both statistical and cost considerations that each individual student take the entire set of items in the assessment. In estimating the proportion of a population who could correctly respond to a population of items, Lord (1962) has shown that a sample with many persons taking just one item each resulted in an estimator with a smaller standard error than one derived from a sample in which fewer persons answered many items. Since such a sampling scheme is not cost-effective since selecting individuals is expensive, a limited number of exercises is presented to each sampled individual.

The basic idea behind Balanced Incomplete Block (BIB) spiraling is to divide the total assessment time into small blocks of items. Each exercise block is then assigned to a number of assessment booklets such that each block of exercises is paired with each other block in some booklet. The booklets are then spiraled so that students in an assessment session are given different booklets. Using BIB spiraling, a large number of booklets must be created, but the interrelationships between objectives may be examined since each exercise is paired with each other exercise in some booklet.

BIB spiraling permits the estimation of correlations between items within a content area and the estimation of correlations of estimates of proficiency between content areas within a subject area. Furthermore, since the spiral design presents each block of items to fewer persons in any school, but to more schools than would a simpler matrix design, the cluster effect is markedly reduced, leading to a sample with high statistical efficiency.

Another advantage of BIB-spiraling is that it balances the position of items across booklets. In a simple

matrix sampling design, every item appears in the same position in a booklet, with the same items always occurring last in a booklet. In cases where fatigue factors are present in students, such a design is likely to underestimate their ability. In contrast, in the BIB-spiral design, block of items are presented in different sequences to different students. As a result of this balancing, results are not influenced by the fact that students tend to perform less well on items occurring at the end of booklets.

Beaton, A. E., Johnson, E. G., & Ferris, J. J. (1987). The assignment of exercises to students. In A. Beaton (Ed.), The NAEP 1983-84 Technical Report (pp. 97-118). Princeton: Educational Testing Service.

Johnson, E. G. (1992). The design of the National Assessment of Educational Progress. *Journal of Educational Measurement, 29,* 95-110.

Lord, F. M. (1962). Estimating norms by item-sampling. *Educational and Psychological Measurement, 22,* 259-267.

Sirotnik, K., & Wellington, R. (1977). Incidence sampling: An integrated theory for matrix sampling. *Journal of Educational Measurement, 14,* 343-399.

Prior to 1984, NAEP used a simple multiple matrix sampling scheme to assign items to students in which a pool of items is divided into distinct booklets requiring about 45 minutes to administer, and all students within an assessment session are given the same booklet. To reduce the cost of administration and increase the efficiency of estimates, since 1984 NAEP has used BIB spiraling. This approach not only allows broad coverage of subject areas but also allows the study of interrelationships among all exercises within and between subject areas while restricting the effort required of any individual student.

Starting in 1988, NAEP has been using focused BIB-that is, each block of items within each subject area has been paired with other blocks within that subject area but not with blocks of items from other subject areas. This means that students are now assessed only in a single subject area. This new method greatly reduces the number of booklets required, but eliminates the possibility of examining correlations between content areas.