6. Evaluation of Vendor Responses
 

Evaluation deals with evaluating vendor responses to determine which vendors to take to the next phase.

Although there may be a preferred vendor at this stage, at least two vendors should proceed to the next phase <7. Making recommendations>.

This phase includes:

MyKnowledge: Gives the overview some insights into developing an approach for evaluation and conducting the evaluation process.

MyApproach: Develops and finalizes the approach to evaluation.

MyDelivery: Provides evaluation scoresheets enabling easy comparison as well as evaluation templates for stakeholders.

MySignoff: Online or downloadable signoff sheet to ensure proper completion of the phase.

Key deliverables from this phase are:

1. Agreed evaluation approach.
2. Functional requirements fit gap analysis.
3. Completed evaluation scoresheets.
4. Completed signoff checklist.

Phase objectives

On completion of evaluation, you should have :

1. Involved various stakeholders in the evaluation process;
2. Reached agreement on the evaluation approach;
3. Scored the proposals based on framework developed in < Request for proposal>;
4. Communicated the outcomes to the successful and unsuccessful vendors;
5. Be in a position to begin more detailed discussions and demonstrations with the selected vendors.

Phase risks

The risks of not completing this section include:

1.     Lack of buy-in and agreement due to poor or inconsistent evaluation approach;

2.         Selecting unqualified vendors, which can result in poor system functionality or implementation services, both of which can impact badly on organizational performance.

The quality of the vendors selected will have an impact ultimately on the quality of the system and the related implementation.

Please work through each section before proceeding to the next section:

Important to note:

Content under each section of the KADS method will differ, based on the phase being worked on. Initially the focus is on learning, moving to delivery and signoff during the life of the project.

What you need to know

You should have received the responses from vendors and be ready to proceed with the evaluation.

The evaluation process is critical to being able to negotiate and make the final recommendation to senior management on which system to procure. Key to the evaluation is defining the evaluation approach and communicating this clearly to all participants. Then evaluating the proposals in accordance with the process and approach.

1. The following document gives some guidelines and suggestions on how the process should work including, developing the evaluation approach, the level of effort and detail, as well as the evaluation process itself. Having a robust process will assist in reaching the right outcome.

The following READ document helps you determine which approach is right for your organization as well as provides the evaluation process.

Name: Evaluating proposals and preferred vendors

Priority: 1

Type: READ

Phase: Request for proposal

Estimated time: 5 minutes

Length: 4 reading pages

At this stage, you should be comfortable with what is required to evaluate the proposals received.

Develop your approach and plans

2. Use the editable User file below to develop the evaluation approach and process as well as the communication. Communicate the approach and process as applicable to the project team and other relevant project participants.

Name: Evaluation approach and process

Priority: 2

Type: User

Phase: Evaluation

Estimated time: 5 minutes

You should now be comfortable with the approach and process for evaluation.

Delivering the approach and plans

3. The following User document should be customized as appropriate for your organization, populated from the functional requirements listing developed in <5 Request for proposal> and used to evaluate the functional capability of each system.

Complete the functional requirements evaluation first. At this early stage, if it becomes obvious that a particular system is not appropriate for your organization the vendor should be eliminated from the process (See step 4)

Name: Evaluation criteria - functional

Priority: 1

Type: User

Phase: Evaluation

You should now have completed the functional requirements document and have a reasonable idea of the system fit to your organizational needs.

4. Where you have clearly identified proposals that cannot deliver on your organizational requirements, communicate with the respective vendor as appropriate.

Assess the number of vendors still in the process, if less three at this stage, more consideration might need to be given to establishing the gaps in functional requirements and whether the vendor should be given a further chance to state their case.

The following template can be used as a basis for communication back with the vendor, informing them they are no longer part of the system selection process.

You will need to decide the form of this communication, i.e. letter or email etc., and what, if any reasons will be given. Generally, vendors will request some feedback. (The more detailed your feedback, the more likely the vendor will respond to tell you they have been misunderstood.) Manage this process carefully to ensure it does not take up a lot of time and resources.

Vendor negative communication letter

You should now be comfortable that the vendors remaining in the process are capable of delivering on your organizations functional requirements.

5. Continue the evaluation using the following user documents to evaluate other factors relating to the proposal and the lifecycle net cost of the system to your organization.

Name: Evaluation criteria - other

Priority: 1

Type: User

Phase: Evaluation

6. Analyze the lifecycle cost of each proposal.

Name: Lifecycle pricing

Priority: 1

Type: User

Phase: Evaluation

7. Hold vendor demonstrations as approproate and review any   uncertainties with respect to function or other requirements. Hold a debrief after each demonstration and get feedback from all participants. Use the following demonstration agenda to give the vendor direction and to manage the time for the demonstration. At least two hours should be allowed.

Demonstration agenda

8. Present your findings to the steering committee as appropriate, using the documents developed above or summaries of these documents to confirm the preferred vendors to begin negotiations with. The template below will assist in the layout and format.

Name: Preferred vendor briefing

Priority: 3

Type: User

Phase: Evaluation

You should now have approval to begin contract negotiations with the preferred vendors.

Sign off on the Phase

9. Update the project plan and project milestone document developed in <2. Planning the project>. Report as scheduled.

10. Complete checklist when you have completed the phase. Please ensure you have ticked all the boxes as yes before you proceed, and print the checklist result and file on completion.

Phase 5 Request for proposal: Completion checklist

1. Has the evaluation approach been agreed?
2. Have the functional evaluation criteria been agreed?
3. Have the other evaluation criteria been agreed?
4. Was the proposals properly evaluated against the evaluation criteria and other requirements?
5. Have you calculated the lifecyle pricing of each proposal?
6. Have the project team and other appropriate stakeholders had proper input?
7. Have you identified the preferred vendors to negotiate with?
8. Have you communicated the outcomes to the vendors who submitted proposals?
9. Do you have sufficient information from each vendor to begin the negotiation process?.
10. 10. Have you completed a detailed review of the preferred vendors and comfortable with the findings to date?

Move onto next phase when ready [ Phase 7 - Making recommendations ]