Kaya Consulting

Assessing effectiveness: It’s a two-way street to sustainable improvement.

Assessing effectiveness: It’s a two-way street to sustainable improvement.

Improving effectiveness – for an organisation, team, or individual – isn’t a linear process. Our fluid five-step methodology is designed to evolve solutions, not impose a formula. That’s why the fifth step in the process, where we assess the effectiveness of our programs and interventions, is so important. In many ways, it isn’t the final step, but the pivotal link in an ongoing process of improvement, for us and our clients.

Determining effectiveness to achieve sustainable improvement
The Discovery Phase within our methodology centres on assessment, establishing the current situation for a client’s organisation and the capabilities of the individuals within it. This benchmarking process – or pre-testing – informs the design and delivery of our solutions. But it’s also the reference point for measuring progress, which we do through ongoing evaluation and assessment.

We call this assessment post-testing, or an after-action review. It’s where we assess the effectiveness of the solutions we’ve delivered using any number or combination of evaluation tools, including:

  • Post assessments and surveys
  • Pulse surveys
  • Proprietary post project questionnaires
  • Post-intervention psychometric assessments

These tools enable us to gather and collate qualitative feedback from the individuals who we’re training and coaching, so we can ascertain what’s working, as well as what isn’t. This information is then compiled into a report that forms the focus of an after-action discussion with our client.

What comes after an after-action report?
The after-action report is clear, transparent and comprehensive. And crucially, it allows us to quickly identify areas where we can augment the delivery of our interventions.

For instance, we might have to shift the emphasis of a workshop from theory to activities that embed the theory through exercises. Or change the format of the material we’re using. Or adjust the consulting or coaching team. Or it might confirm that the program we’re delivering is working as intended and planned, in which case we’ll continue and review again at regular intervals.

And, of course, once a project is complete, the after-action review process enables us to chart the progress made by our clients’ employees, including the real-world impact and performance improvements that flow on from our programs.

An example: A Development Story
Our case studies and the outcomes our consultants achieve bear testament to the effectiveness of the after-action review process. Take our Development Story, for example.

It’s all about our work with a major global accounting firm in South Africa, which included a program of workshops and interventions for students looking to join the firm as accountants after completing their university studies. The success of the students hinged on a number of factors, chief among which was passing the National Board Exam.

The problem
Pass rates for students were troublingly low, especially among black bursary students. With many coming from disadvantaged backgrounds, many were struggling to cope with university life, which ultimately manifested itself in a poor exam pass rate.

We therefore developed a program of EQ (emotional intelligence) workshops and interventions, all aligned with our clients’ capability framework, that supported students and equipped them to thrive in their new lives at university. The program comprised two workshops per year, with students attending six workshops during their three-year university course.

During the six years that we ran the program, we conducted our regular after-action review process, adapting and improving our interventions where required. And the results, as outlined in the final review, speak for themselves.

The benefits for students
When comparing students that attended our workshops with those who didn’t, our client found that:

  • 93{be38fd1e2c946a347db4d7316b241dce4b842100e7b38236661610f0dce6def9} of those who attended our workshops passed the National Board Exam
  • Only 43{be38fd1e2c946a347db4d7316b241dce4b842100e7b38236661610f0dce6def9} of those who didn’t attend our workshops passed the exam

Furthermore, among black bursary students:

  • 82{be38fd1e2c946a347db4d7316b241dce4b842100e7b38236661610f0dce6def9} of those who attended our workshops passed the National Board Exam first time
  • Only 57{be38fd1e2c946a347db4d7316b241dce4b842100e7b38236661610f0dce6def9} of those who didn’t attend our workshops passed the exam first time

The benefits for our client
Beyond the positive impact on pass rates and the lives of the students who took part in our program, our interventions delivered a host of practical and financial benefits for our client too.

Our graduates were better prepared for the world of work and more able to add more value in the workplace once they joined the firm. Moreover, the improvement in exam results helped elevate the firm as an employer of choice, enhancing graduate retention and thereby minimising the costs associated with employee recruitment and development.

Find out more about the flexible five-point methodology we’ve developed to analyse and improve organisational effectiveness. Read the real-world stories that demonstrate the value of organisational psychology, or talk to us about your unique workplace and organisational effectiveness challenges.

Jan Sipsma is an organisational psychologist and founding partner of Kaya. With over 20 years of international experience as an organisational effectiveness consultant, he specialises in strategic planning, organisational architecture and design, change management, capability and performance enhancement and the identification and development of leadership. He has a Masters of Commerce (Industrial Psychology) cum laude and is a registered member of the Australian Psychological Society.