The University is committed to the pursuit of excellence in all aspects of its academic, scientific, clinical and administrative work. To understand the requirements of our students, customers, employees and constituent groups, we must assess their needs, experiences and expectations. An effective way to accomplish this is through the use of surveys. Surveys can be used to guide change, validate observations and determine the extent to which constituent groups are satisfied. Properly designed and implemented, surveys can influence the University's decision-making process, enhancing the level and quality of service we provide.

Face-to-face interviews, focus groups and telephone surveys are some methods employed to gather data. This article focuses on the written survey, which, because of its relative low cost, is believed to be the most commonly used. There are four areas of concern when developing a survey instrument: planning; question development; pilot testing; and logistics.

Step 1. Planning the Survey Effort: The most important aspect of survey design is clarifying goals and objectives. Establish the purpose/aim of data collection, define the target population and determine the demographic factors, which will be important to the segmentation of responses. Equally important is determining specific areas of exploration and the mode of analysis.

Frame survey goals and objectives in a well-defined statement of purpose. To formulate goals, determine how the data will be used, what they will help accomplish and how they will enable the implementation of change or validation of assumptions and observations. Be clear about intended results, openly communicating them to others involved in the planning effort.

Define the target populations and/or identify key group(s) from whom data are required. This is critical to the development of appropriate and easily understood questions. Assess the similarities and differences in your target population and how to characterize them. Determine how to capture these elements in the survey. This demographic information will help identify response trends in the larger group. For example, the school, course of study, campus, student class, and semester may make a difference in the responses to a student survey. Determine the sample size. To make meaningful inferences from the data, the sample must represent the target population and the number of responses must be adequate to support statistically valid conclusions.

The identification of high-priority information needs is critical to refine the survey development process. Identify key issues for analysis, avoid what is just interesting or good to know. Develop succinct questions and consider if confidentiality will be important to the respondents. In some cases, surveys conducted by an outside source can extract more candid responses.

The method employed to interpret responses is central to the entire survey effort. The mode of analysis will help determine satisfaction levels, verify assumptions/observations, identify trends and highlight areas where improvement efforts are most needed. Computer software programs are available to assist in interpreting data. Thoroughly research software programs before constructing the actual survey questions to be sure the program's capabilities align with the format of the survey instrument.

Step 2. Constructing the Survey: Develop a list of all areas the survey will examine. Each area represents an element for analysis and is called a survey dimension. If the survey targets the quality of service, areas for exploration might include availability, timeliness, responsiveness, professionalism, completeness, and overall satisfaction. If assessing employee attitudes and perceptions, survey dimensions may include organizational processes, employee morale, problem solving, decision making and overall job satisfaction. Specific inquiry in these areas will enhance your ability to implement change or validate observations based on the collected data. Respondents generally expect their feedback to be used. It follows, then, that survey questions should focus only on what is within your purview to change, accomplish or validate.

There are two types of questions you can use on written surveys -- open and closed-ended. Open-ended questions allow respondents to answer questions in their own words, which can be a rich source of information. The disadvantage to open-ended questions is deciding how to analyze the responses. Any questionnaire consisting of open-ended questions requires thematic analysis, and bias can inadvertently be injected. Multiple choice, rating scales and yes/no answers are considered closed-ended questions. These questions offer respondents specific choices and usually result in a quick response. However, unless every possible option is provided in the selection matrix, the validity of the information may be compromised. Good results can often be achieved by combining both open and closed-ended questions on the survey. Avoid the use of leading and/or compound questions or statements, acronyms, jargon and double negatives, which can result in unusable responses. To convey impartiality, begin all questions using words such as: how, what, when, why or do.

Step 3. Testing the Survey: Regardless of how confident you feel about the survey instrument, pretest or pilot it. The entire survey, including the instructions, must be reviewed. This will help determine the extent to which the survey will perform its intended function. Test the instrument using representatives from the target population. A small group is considered the best forum to confirm the clarity of the instructions, identify questions that are inappropriate or poorly worded, determine the appropriateness of the survey format and the length of time it will take to complete it. The input of others is essential to ensure the best possible survey instrument and reliable results.

4. Logistics: How many surveys are needed, how many individuals must be contacted to get the number of completed surveys required, and how best to reach them? Who will enter the data, how long will it take, where will the information be received and where will it be stored? These questions should be addressed before conducting the actual survey. It is important to note that during the survey development process many steps are performed concurrently, requiring an efficient and effective planning process.

Careful planning and implementation will significantly impact the survey effort; less time will be spent in analysis and interpretation and more reliable results will be obtained. The Department of Planning can refer you to experienced professionals to assist your survey effort. For assistance call Mr. Bruce Kenney at (973) 972-3814 or e-mail kenney @umdnj.edu.