Valid Feedback

Q. As we have used different surveys in our company, there are always some who question the results. They complain that the questions don't apply, or that the survey does not provide "valid" feedback. How can I reply to the nay-sayers, and ensure that our survey instruments provide accurate feedback?

A. The credibility of the survey instrument can make or break a feedback process. If participants don't believe that the survey or the feedback is accurate, the entire process is jeopardized. To ensure success, the survey instrument must be valid and reliable for your organization.

Without going into the statistical detail behind validity and reliability of surveys, it is important to understand that the feedback provided is about perceptions of how a person performs, or about a person's behavior. Therefore, it is critical that the assessment be designed to help the respondents provide feedback that allows them to best describe the way they view a particular person's behavior. The type of survey chosen is very important to this process.

We group most surveys into two categories- Standard and Customized. A standard survey is often based on a model or theory, contains a static (non-changing) question set, and is usually branded by the company that created it. A customized survey is just what the name implies-- the question set is customized to align with the leadership competencies or objectives of the individual organization. There are advantages and disadvantages to both:

    Standard Surveys

    Advantages
    • Often based on a single model or theory
    • Can provide for better large-scale norming (industry comparisons)
    • Facilitation or certification materials are often available
    • (Perceived) greater validity and reliability possibilities


    Disadvantages
    • Generic questions may not measure what's critical to your organization
    • Assumption that Job A is the same as Job B, and that Industry X is the same as Industry Y
    • Inflexible
    • May be based on flawed models


    Customized Surveys

    Advantages
    • Allow complete flexibility
    • Can be tailored to measure what is important in the organization, industry, and function
    • Can be more culturally appropriate
    • Greater buy-in from participants

    Disadvantages
    • Usually are not associated with a "leadership theory" or model
    • Limited norming possibilities (industry comparisons)
So which is best? Based on our experience, we recommend well-designed customized assessments. Customized surveys ensure greater buy-in to the process because employees know that the survey was tailored to their organization. Participants have greater confidence that the feedback is accurate and usable for action planning.

Confidence in the survey is the first step to implementing a successful feedback process. If participants feel that the feedback is valid and reliable, they will have a greater desire to participate in the action planning process and follow through on commitments. Remember, the process is more important than the instrument, but people need to believe in the instrument before they will participate in the process.