Talent Management
Applicant Tracking

How To Test Your ATS Like An Applicant

Published by in Applicant Tracking

I’m forever going on about how applicant tracking systems are the worst.

(Yes, I know I also write about how great I think they are. It’s a love / hate relationship kind of thing.)

The main problem I have with applicant tracking software is that they are just programs. And a computer program will only ever be as smart as the people programming. And while, as a whole, that’s pretty dang smart (dude they made a computer do their bidding by typing at it, which is awesome), there will always be limits.


Programs are literal. Give instructions to a human and they can interpret. Tell a program what to do and it can only do that, nothing else. Since us humans aren’t so literal, this literalness comes as a surprise to us when our programs are.

Like I said. Programs aren’t smart.

What does this mean for your ATS? Your program has bugs. Not probably, not maybe, definitely has bugs. And you don’t even know they’re there. But your candidates know they’re there, because they’re the one dealing with them, and they might not know how to tell you, or that an issue was because of a mistake on their end or yours.

The only way to combat this is to become the applicant and test your system as if you were using it to apply for a job. How can you perform this test most effectively?

1. Make a scorecard

A self-designed scorecard is a great way to keep your thoughts organized and compare different software on similar metrics. Giving yourself some form of standardized scoring system (perhaps a 1-10 ranking scale) is a great start. You should also have notes on each product that will help you keep them differentiated and organized, and make sure to include a space where you can make comments on each feature you explore. That way, you’ll be able to remember why you gave one product a “2” and another a “9” three months down the line. It is imperative that you have a clear, uniform way to rate your experience and express your thoughts on the application process as you move through your card.

I’ve made a general sample scorecard for you here. I included boxes for notes on ease of use, how clear the instructions are, how frustrating the overall system is, and how long it takes to complete. These factors matter, because candidates often give up on their applications if they encounter tech issues, or if the system is too long or complicated.

Frustrated applicants leave. You don’t want to make good applicants leave.


Feel free to use my chart as a jumping off point, but consider adding other metrics for what’s important to you. Some to consider are design, branding, and language used in the ATS. And it doesn’t matter if you used a numbed scale ranking or notes or a mix of both to rate your software’s performance, as long as it makes sense to you.

2. Design the perfect resume

Don’t look at your keywords. You might not even want to look at your job description. Just create a resume (from scratch, please) for the candidate you would love to see.

Don’t be afraid to fill in fictional details such as the clubs and sports they did in college or awards they might have gained in graduate school. Don’t only focus on their workplace achievements. Add details that would indicate culture fit.

Create the articulate, well-rounded, brilliant polyglot you’ve always wanted to interview.

Got that? Run the resume by your hiring team and see if, just by a cold-read, this is someone they would like to hire. Confirm this fictional person is perfect. This is the resume you will be using to test your ATS.

Why? Because the perfect candidate for a human might not be what your ATS thinks of as a perfect candidate. In fact, this ideal resume might get rejected outright by your ATS. It’s happened before and is a great indicator you have a problem.

3. Test Your ATS

If you’re done prepping everything, you’re ready for the big moment. You need to logout of any administrative functions and create a tester dummy account on your ATS. And then you upload your resume and fill out the forms. Simple as that.

Go through slowly and methodically, using your scorecard every step of the way. This is where the space for taking notes comes in.

4. Test again

You’ve filled out your scorecard, you’re feeling solid about your candidate experience. Great! Now test again!

This time, use the resume of one of your existing employees in the same position. Why bother? Because you know they’re great, but your ATS probably doesn’t. If your ATS rejects your own top talent, you need to find out why and look into fixing whatever is broken.

5. Revise

Now that you know how your ATS feels from the other side, figure out how to improve the experience. This might mean switching to a new software. This might mean streamlining the process. This might mean switching the keywords you look for. Whatever it means for you, keep testing until that ideal resume gets through. You need to be absolutely sure your system works the way you want it to, and continuing test and tweak until you have the results you need is the best way to make sure that happens.

Other tips?

Have you tested your ATS before?  Did you take a step I didn’t detail? Did you make sure to share the results you got in a review so other people using the software will know what you found? Sharing these results will mean a lot to other people getting your program for the first time! Tell me more about how you feel in the comments.

Looking for Applicant Tracking software? Check out Capterra's list of the best Applicant Tracking software solutions.

About the Author

Halden Ingwersen

Halden Ingwersen

Halden Ingwersen is a former Capterra analyst.


Comment by Saïd on

Thanks Halden! I like the love / hate relationship statement. I do have the same relationship with recruitment, which motivated me to build a great Applicant Tracking System that people will only love !

Keep up sharing great content!

Comment on this article:

Comment Guidelines:
All comments are moderated before publication and must meet our guidelines. Comments must be substantive, professional, and avoid self promotion. Moderators use discretion when approving comments.

For example, comments may not:
• Contain personal information like phone numbers or email addresses
• Be self-promotional or link to other websites
• Contain hateful or disparaging language
• Use fake names or spam content
Your privacy is important to us. Check out our Privacy Policy.