Common Questions and Answers
A: When you have a single PSU in a stratum, you cannot calculate the within-stratum variance. We use the heuristic approach of assuming that the within-stratum mean is the same as the overall mean, enabling AM to make the calucation.
A: To get a matrix of bivariate correlations you select "correlations" from the statistics|basic statistics menu. Drag multiple variables into the dependent variable box, and you will get a matrix of correlations among these variables. In this procedure independent variables define subgroups, and correlations are reported within each subgroup.

The correlations procedure does not currently produce standard errors or significance tests. I know that it is inconvenient, but running a bivariate regression will give you the same significance test. I suggest the following approach:

1. run a correlation matrix.
2. Identify the bivariate correlations of interest (say, between x and y)
3. select statistics|basic statistics|robust regression
4. make y your dependent variable and x your independent variable, and appropriately specify the weight, strata identifier and cluster identifier
5. press go.
6. repeat for all pairs of variables of interest

Please note: the latest version available on the web allows you to copy, modify and re-run an analysis. Right click on the completed run icon for this option.
A: AM Currently supports balanced repeated replication and several jackknife replication methods. please see the documentation.
A: The plausible values procedures, as with most procedures in AM, use a variance estimator based on a Taylor-series approximation of the sampling error. The estimates are as good as the jackknife estimates based on replicate weights. If specified correctly, your standard errors will be right.

The Taylor series estimates require that you specify the variable identifying sampling strata and the variable identifying primary sampling units in addition to the weight variable. In the NALS data set, these variables are named TSLVSTR and TSLVPSU, respectively. You must also specify the weight varialbe, which is named NFLWGHT.

If you right click on these variables (one at a time) and select "Edit Metadata", you will be offered a chance to specify the "design role" of the variable. This allows you to set the default weight, strata, and cluster identifiers, which will be automatically included in each analysis, unless you specify otherwise. You must save the dataset to make these changes persist across sessions.
A: Currently, AM tables procedures do not produce marginal estimates in the same run as cross-tabs. You can accomplish this by doing two separate runs--first do a run with no crossing variables, then do a run with the crossing variables that you want.

If you want to make sure that the runs are all based on the same data (missing data might cause values to be omitted) you can do the cross-tab first--then, from the Completed Run Queue, right click on the run icon and select "Create Sample Indicator." This will add a variable to your database that takes on a value of 1 if the observation was used in the crosstab. You can then filter the data based on this variable.
A: 1. AM does not currently provide significance tests for bivariate correlations. However, if you want a test of the significance of the correlation between two variables, you can run a bivariate regression and check the significance of the coefficient of the predictor variable.

2. When you use weights for most datasets they weight up to the population. The weighted N then represents an estimate of the population size, rather than a sample size. The standard errors and significance tests are all appropriate, so there is no reason to "normalize" the weights. Nothing that you are doing is wrong.
A: It sounds as though you are trying to download the patch, rather than the main system. I realize that the website is not that clear on this issue. You should click on the computer, and it will download a file of approximately 10 megs.

Please let us know if this does not work.
A: You can import data from a fixed-format ascii file. Fixed-format means that all the variables are always in the same column. To import, you will need a dictionary file and your data file. The syntax for the dictionary file is in the help system under Getting Started.

At a minimum the dictionary file requires the following information:

FILE = "filename" (put the real path and name of your files here)

VARIABLES

name1 1-2 type=r
name2 3-3 type=b
(names should be the name of your variable, next give the start and end columns, and type should be "b" if the variable is an integer in the range of -126 - 126, and "r" otherwise. You can also add variable labels, formats and other information. Please see the help system)
A: 1. The singular matrix message is valid. You probably had variables that were perfect linear combinations of one another. For example, if you put in three dummy variables from a variable that had three levels, plus a constant, the sum of the dummy variables would equal the constant.

2. The error message is valid. All of the variables that you would like to correlate should go in the dependent variables box. The independent variables should be grouping variables,and you will get separate variance matrices for each group. I believe that the maximum number of groups is currently set to 50.

3. I assume that you are using netscape 7. That automatically uses a different java plugin than the one that installs with AM. We will have a fix out soon, but in the mean time, you should use internet explorer (my apologies).
A: absolutely. You use it with health datasets as you would any other dataset. If you have any specific questions about how to do this, please feel free to e-mail them to me.
A: The released version of AM does not estimate IRT models, so you should use Conquest, Parscale, Multilog, or one of the others for now.

We are currently testing an IRT model, which should become available this fall.
A: 1. The log likelihood increases as more predictor variables are added, in both SPSS and AM. AM does not currently report the beginning log likelihood statistics (associated with the null model that includes only an intercept). The single figure is used only for comparisons across models. However, most data does not use a simple random sample, so the log likelihood (from either AM or SPSS) is of little use in testing model specifications.

2. AM does not output odds ratios, though they are a simple transformation of the coefficient estimates. If you want them, output to a spreadsheet and do the calculations there.

3. All results from complex samples are based on asymptotic theory, and are valid for large samples. AM reports z-statistics for this reason. In SPSS the t-statistics are not correct if the sample is complex (i.e., unequal probablity of selection, stratified, or weighted)
A: This problem can occur with the Windows 98 operating system (I don't know about older ones).

If the user is disconnected from the network, but retains their network configuration, the operating system thinks it is still attached to a network. AM tries to talk to the output browser through a socket, and the system stalls out waiting.

To fix the problem, go to the operating system and get a command line. Then type:

ipconfig /release

When you reconnect to the network, run

ipconfig /renew
A: Unfortunately, that is currently the case. The output system uses frames-it has a simple java program in the top frame (the one with the AM logo). When you save the output, it saves the page that indicates the frames. Anything opening it needs to be "smart" enough to go find the frames.

Fortunately, there is a simple solution. At the beginning of each session, set your output file by selecting "Results | Web Browser Output |Select File" before you start analysis. All output wil go to the file that you specify, and that file does not use frame--it's just the output.

If you forget to do this, you can specify the output file at any time, then go to the Completed Run window, right click on any runs you'd like output from, and select "Results". The results will be written to the newly designated output file and will be displayed.
A: This means that the standard errors could not be estimated, most likey because of perfect collinearities in the model. The most common error that causes this is creating a set of dummy variables, and including all the dummy variables plus a constant term in the model. When you do this, the sum of the dummy variables equals one (because one of them is always equal to one, so the sum always equals one). Try eliminating one of the dummy variables.
A: 1. Normally, a multinomial logit model assumes that equal variance across subjects. However, in a choice model, some people may be better informed than others. Hence, we would expect their choices to be more predictable than their less-informed counterparts. The variance predictors allows you to model this.

2. While I know that some packages offer various measures of explained variance for ordinal variable models, I never quite understood why. The variance of a continuous variable carries some information, but for a binary or ordinal one it does not. We will at some point introduce some measure of the extent to which the model reduces prediction error, but we have not yet.

(Sorry for the delayed response. I was unable to check the site for questions over the weekend because the hurricane took out my power)
A: yes there is, and it is quite easy to do. Run the analysis of interest. Right click on the Completed Run Icon. One of your choices is Create Sample Indicator. When you select that, you will be prompted for a variable name to hold an dummy variable, which will take on a value of one for those cases that were included in the analysis.

You can then set a filter (select Data | Filter Observations) to limit analyses to that subset. Suppose you named your sample indicator FOO. the expression that you would type into the filter window would be (FOO=1). The filter will persist for all analyses until you clear it or close the dataset.
A: Test
A: The documentation (and browser output system) requires that the Java plugin successfully install with the software. For users of Windows NT, 2000, and XP, this will require administrative privileges.

If you do not have administrative privileges on your machine, the java plugin will not install and the neither the help nor browser output will work.

The solution to this problem is to have a system administrator reinstall the software for you.

If this is not the problem, or if this does not solve the problem, please e-mail me and we will sort this out.
A: The manual that is online is also downloaded and installed when you install the program. Open AM and press F1--this will bring up the manual in your browser.

Also, note that the manual is the context-sensitive help system, so if you press F1 while a dialog box is open or a menu item highlighted, the manual will open to the relevant page.

We had a user's manual that is now out of date. please let me know if you'd like a copy--it might be of some help getting started. You can send an e-mail to am.air.org to request it.
A: The denominator degrees of freedom in the (adjusted) Wald tests are calculated as the number of PSUs (clusters) less the number of strata (unless a stratum has a single PSU), less the number of regressors.

Please see the Help system for more discussion. When the dialog box is open to do a Wald test, press F1 for help on the Wald test. You might be particularly interested in the "Details" page. The references page provides some helpful citations.
A: Yes. After you run the frequency table, right click on the completed run icon in the completed run queue. One of your options will be "show totals." Click that and you will get a table of totals rather than percents.
A: We can add that feature in the next version. In the mean time, it is simple enough to create dummy variables that take on a value of 1 if the respondent is in the literacy level, and zero otherwise. If you do this for each plausible value, you can run the plausible values means procedure.
A: The dependent variable in the frequency procedure is the column variable. You must specify a column variable.

For the weight, strata, and cluster fields, you must drag and drop them.

Alternatively, you can right click on the appropriate variable in the variable list, and select "Edit Metadata." Fill in the appropriate "Design role." For example, right click on the weight variable and select "weight" for the design role. These will then load automatically when the statistical procedure dialog comes up.

About the help (and output) system: You must have administrator rights on your machine when you install it. The problem may occur because the java plugin (which requires administrative rights) did not install correctly.
A: we have improved the import capability. you must also download the DFTransfer component. Then, you will be able to import both .xpt and native SAS (.sas7dat) files, along with 150 other file formats (pretty cool, huh?)
A: The answer to the first question is no, not directly. I may add that capability in the next release if there seems to be call for it. You could obtain the estimate by running the logit separately five times, and combining the results as described in the PIRLS technical manual.

To run regression models separately by groups, you currently have to set a filter to select one country at a time, and run the model separately for each country.

As an aside, you may be interested in the Demonstration site for our new on-line analysis too, AIR Lighthouse. The sample site (http://lighthouse.air.org/timss) gives you the ability to analyze TIMSS and PIRLS data, and will give you separate analyses by country without any real effort.

Please let me know your thoughts about AIR Lighthouse. You can write back. Thanks
A: The multinomial logit procedure appears on the menu with two variance estimation options: one using replication methods, the other using a taylor-series method (under Basic Statistics). The appropriate choice depends on the data set that you are analyzing.

- If your data set includes replicate weights, you should probably use a replication procedure.
- If your data set includes strata and PSU identifiers, you should probably use a Taylor-series method
- If your data set has neither, you probably have a simple random sample, or a data set that was not properly constructed. If the sample was drawn as a simple random sample (possibly with unequal weights), you can use the Taylor-series procedures.

You can specify the weights in one of two ways:
1. Drag them into the appropriate box from the variable list; or
2. right click on the variable in the variable list. To specify an overall weight, select "Edit Metadata" and press the "weight" radio button under "design role." To specify replicate weights, select "Specify replicate weights," and drag the appropriate variables into the dialog box that pops up.

The difference between the replication variance estimation methods (JK1, JK2, etc.) has to do with how the weights are constructed, and how the variance calculations are done. You can find the details in the help system, under Tools|Variance Estimation|Jackknife and Tools|Variance Estimation|Balanced Repeated Replication

On the gender variable--the answer depends on how you want to use it. Is it the criterion or dependent variable? If so, drag it into the dependent variable box. If it is an independent variable, drag it into the independent variable box. If you believe that prediction variance differs between groups, drag it into the variance predictors box.

I hope this helps.
A: To estimate the variance within a stratum, you must have at least 2 primary sampling units (PSUs). If there is only one, AM will will estimate the variance around the overall mean, rather than the stratum mean.

you can right-click on the completed run icon to get a list of single-PSU strata. As long as this list matches what you believe is in your data, your estimates should be fine. Essentially, the approach that AM uses treats those strata as though they are from an unstratified sample.

Please feel free to write back if you would like more information or some citations.
A: The reason that you get the "browser loading slowly" message is exactly that--the browser is taking a long time to load. A java applet within the browser opens a socket to recieve information from AM, and in doing so, queries the network. Hence, if you network is slow (possibly due to heavy traffic), the browser can load slowly.

If you get the picture of the coffee-cup and sun in the browser, you simply need to click the "continue waiting" button one more time, and your results will appear. After that, do not close the browser--subsequent output will be sent more promptly.

If you do not get the coffee cup, but instead get a "missing picture" icon in the top frame of the browser, you have not installed the Java Plugin, which loads with AM. Reinstall that plugin, and you should be OK.

If neither of these fixes the problem for you, please write me and we'll pursue this further. In the mean time, select "text output" instead of HTML output from the dialog box with which you specify the statistical run-- the results will be written to a text file rather than HTML, and will come up in a text editor. It is a bit ugly, but at least you'll have results.
A: It sounds like you did not completely install the software. After you install the AM, wait a minute or two and a "java plugin" will start to install. You must accept the license agreement and complete that installation to use the web-browser output.

At some institutions, users do not have administrative rights to their machines. If that is the case at your institution, the java plugin will not install, and you must ask someone with adminsitrator privledges to install it for you.

If you have further problems, please contact me.
A: Hi,

I apologize for the expression parser--we will replace it as soon as we can. You must use mathematical expressions, rather than logical ones.

Comparison result in a 1 if true and zero otherwise. therefore, to create a variable that is

var1 >= 3 and var2 >=3

you would enter

(var1 >= 3) * (var2 >= 3)

which only evaluates to one if both expressions are true (otherwise you are multiplying by zero).

If you want to do the same thing, but with either one condition OR the other, you would use

(var1 >= 3) + (var2 >= 3)

Let me know if you have any further problems.
A: You will need to have adminstrative privileges on your computer. The program installs along with the Java plugin for your browser. (AM presents output and the help system through the browser, and the plugin is required to make this work smoothly). Therefore, you should probably ask someone in your IT department to install it or give you administrative rights.

If you have any further trouble, please contact me.
A: The software does not currently import Stata version 8, and it is not on the import menu. However, it does import Stata version 7. In Stata Version 8 you can save your file as a Stata Version 7 file by issuing the command "saveold"
A: It certainly is possible. I believe that dataset includes both replication weights and strata and cluster identifiers. If I remember correctly, the latter are VSTRATA and VREP. I don't remember the name of the overall weight variable.

You can use the replication procedures with the replicate weights, or the taylor-series procedures (mostly under "Basic Statistics" on the statistics menu) by specifying the appropriate "design variables." Design variables are the replicate weights for the replicate procedure, or the strata, cluster, and weight identifier for the taylor series procedures.

Please e-mail me if you have any further questions.
A: Currently, AM offers this option for some of the linear regression procedures. Right-click on the completed run icon to do create the predicted value. You can then use one of the export features (File|Export) to explort the observation-level file.

e-mail me if you have questions.
A: Yes. However, the parser is not very friendly--you have to use mathematical operators. Each conditional expression (such as RPTSAMP=1) results in either a 0 or a 1. Non-zero values are kept in the sample.

If you want RPTSAMP=1 AND SCHTYP=1, you would enter
(RPTSAMP=1)*(SCHTYP=1), which would result in a zero if either were true. Similarly, addition accomplishes the work of "OR"

We hope to put a much more powerful and easy-to-use programming language in a subsequent version.

Please e-mail me if I can be of any more help.
A: Or is +. Each expression evaluates to a one or a zero, so a variable that equals 1 if the x = big or x= tall and zero otherwise would be

((x=tall) + (x=big) > 0
A: You are not doing anything wrong. You cannot currently save graph specifications--you can save the graph itself in a variety of formats, but you must re-run it if you close the database.

Also, for statistical models, you must explicitly save the .sAM file by right-clicking on the completed run icon, and selecting the appropriate menu option--it does not save automatically when you save the dataset.

There is some good news: we are currently designing a programming language for AM that will not only allow you to write and save specifications, it will also provide access to AM's internals--maximization routines, matrix math, etc. This will make the program much more flexible. As always, it will remain free.
A: absolutely. You may even find it a bit easier to use. If you have any questions about how to use it, please feel free to e-mail me.
A: Hi,

That message indicates that the regressor matrix is not of full rank.

When using dummy variables, the most common cause of this condition is including one variable for each category, rather than excluding one category. When you do this, the set of dummy variables is perfectly collinear with the constant term. Instead, you should exclude one category and interpret the coefficients associated with the others as the difference from the excluded category.

Please write me if you have any further questions or if this is not clear.
A: The sectioned density plot does not display the minimum and maximum values on the bottom bar. Each bar (or _section_) appears at a certain relative freqency threshold. The more sections, the smaller the increment. The darkest bar ends where the first relative frequency increment is passed. Very long, thin tails only show up when you use an unreasonably large number of sections.
A: You can drag as many variables as you like into the dependent variable box in the descriptive statistics procedure. This should give you the table you want.

AM does not generate standardized coefficients. If you really want them you can output to a spreadsheet and calculate them there.

e-mail me at this address if you this does not answer all of your questions.
A: yes. Unfortunately, SAS was not fothcoming with descriptions of their format catalogue data format. Therefore, you first have to save the formats as a SAS dataset:

proc format data=XXX cntlout=yyy;

Then, when you run the import you will be promted for format files. Add the SAS format data set file to the list, and it should work.

Please e-mail me at this address if you have any further problems.
A: Hi,

The import function uses third-party software that we have licensed for distribution with AM. We plan on updating it within the next year; however, you can include your current formats. To accomplish this, run:

proc format library= cntlout=

to create an output data set (make sure it is a version 7 dataset you create). You can import formats from this library.

Let me know if you need more help. e-mail me directly at the address above.

Thanks
A: AM does not currently calculate Cronbach's alpha. That statistic is typically not reported with a standard error, so you will have no problem if you compute it in SPSS, SAS, or Stata.

Please write back if you think that this would be generally useful enough to warrant inclusion in a future release.
A: it sounds like the browser output system did not install correctly. Some users have been having this problem recently because their institutions do not give them sufficient rights to install the software correctly. We are working on a solution for the next release, so that users without administrative rights on their computers will be able to install the software themselves.

In the mean time, the solution is to ask your system administrator to re-install the program for you. It is usually a good idea to check that the browser output and help system function properly before letting the system admin leave your office.

Let me know if this does not solve your problem.
A: No problem at all.

Just use mathematical notation. So, for example, if you want all cases where A > 0 AND B > 0, you write

(A > 0)*(B > 0)

If either condition is not true, it evaluates to 0, and the entire expression becomes zero, excluding the case from analysis.

If you want to A > 0 OR B > 0, you write

(A > 0) + (B > 0)

Please let me know if you need any further help.
A: I am sorry that you are having trouble. We do not have a current user's guide, but all the documentation is available online. You should be able to access it by pressing F1 from within AM--it will come up in your browser. If it does not, you may have to reinstall the software, making sure that you have administrative rights on the computer first.

The manual can also be browsed online at am.air.org (select "Manual" from the home page).

If this is not sufficient, we do have a very old version of a user's guide that may be of some limited help. If you'd like a copy, please e-mail me at this address.
A: 1. When a sample is stratified, clustered, or weighted variables indicating the strata, cluster (psu) and weight typically appear on the data file. This information about the sample design enables AM to accurately estimate the standard errors.

2. ANOVA can be implemented using the regression procedures. Run a regression with a dummy variable representing each group (excluding one group, which is represented by the constant term.

3. Covariances of the estimates are available by right clicking on the completed run in the Completed Run Queue (top right window, select menu option). The descriptive procedure will give you correlations and variances, from which you can calculate the covariances if you need them.

4. confirmatory factor analysis/ structural equation modeling? AM can do a limited set of specific models, but does not currently have a general capability to estimate these.
A: Yes. Check the PIRLS manual or contact NCES or the International Study Center at Boston College to find the names of the design variables (strata, cluster, weight).

Also, you may want to check out http://lighthouse.air.org/timss where you can analyze the data on line. Let me know what you think after you try it.
A: The many new security features in windows has made this a relatively common problem. Here are the most likely culprits:

1. if you don't have administrator rights on your computer, then the help applet did not install properly. Have someone with administrator rights install it.

2. Your browser may be not allow java to run. Go into your security settings and tell it to allow java applets to run without prompting. In the default configuration, IE will give you a barely perceptible prompt asking if you want to allow the program to run, simply telling it "yes" works if this happens.

In either case, the browser output system will nto work either until you get it straigtened out. Please contact me at this address if you have any further trouble.
A: It is simple. Run descriptive statistics. Your dependent variable will be the one whose mean you are interested in. Your independent variable will be a categorical variable that defines the two groups that you want to compare (no filters necessary).

After you do the run, right click on t-test. Then, click on the first mean. When you click on the second mean you will see the t-test result.

Please write me at this address if you have any further difficulty.
A: Hi,

Sorry that it has taken so long to get back to you.

The problem that you are encountering is that your machine is running out of memory. AM should not crash when this happens, but the current version does (sorry .. a fix will be available soon). The reason that this happens is that the table that you are trying to create is simply too large. I'll bet that the fourth variable that you are adding has very many unique values, leading to matrices that are enormous. Try creating variables that have just a few categories.

The new version should be available in a couple of months, and this problem will go away.
A: Hi,

AM uses third-party software for the import procedures. I have imported very large files, but never 28 million records. Is this data that you can send to me? If so, I can try to track down the problem.

please respond to this address.
A: Question 1: AM can do Sectioned Density Plots, which replace box-plots and histograms. It is not clear whether you want to graph the weights themselves, or have them reflected in the data that you graph. If you want to graph the weights, you can do that from the graph menu. If you want graphs that reflect the weights, run the replication procedure of interest (say, the descriptives procedure), then right click on completed runs and request a bar or line chart.

Question 2: The export procedures will export any variables that you request, including the BRR weights.
A: 1. Currently AM does not have a test to determine whether the pattern of data across cells is likely to be random (what the chi-square would test). A traditional chi-square on a complex sample would not be an appropriate test. I'll put such a test on a "to do" list to add to the program.

2. An ANOVA can be accomplished via the regression procedures. Create dummy variables for each category or interaction of interest and introduce them into the regression. Please note that usual F-tests will not be valid in a complex sample, but you can use the Wald test in AM.

Please write me back if I can be of more help.
A: The current parser only uses mathematical notation. Your code should look like this:

(C5SSCALE >0) * (C5R2MSCL >0) *(C5R2RSCL >0)

If all conditions are true, the result is nonzero (1 in this case) and the case is retained. Otherwise, it evaluates to zero and the case is filtered out.
A: Sorry for the delay in getting back to you. You should select

Statistics|Basic Statistics|Correlations

from the main menu, then put all of the variables for which you would like correlations in the dependent variable box.

If you'd like this broken down by subgroup, put the categorical variables in the independent variable box.

Please write me at this address if you have further questions.
A: There is no F-test because that is only appropriate in a simple random sample. The Wald test statistic will serve this purpose in a complex sample.

Please write back at this address if you have further questions.
A: AM will run without adminsitrator rights, but someone with administrator rights must install the software. Your lab administrator can probably accomodate.