The Capterra Shortlist Methodology

Capterra profile picture
By Capterra

Published
7 min read

The Capterra Shortlist Methodology

Current Methodology: January 2021 - Present

N.B. In our ongoing efforts to improve the user experience of the Capterra Shortlist, we’ve made some enhancements to the visual representation of the Capterra Shortlist, effective July 7, 2023. (The methodology and scoring have not changed.) Specifically, the Capterra Shortlist graphic no longer labels its four quadrants, which were previously labeled Top Performers, Emerging Favorites, Established Players, Noteworthy Products. This better reflects the exclusive nature of the entire Capterra Shortlist and ensures the featured products are the focus of the Capterra Shortlist.

The Capterra Shortlist identifies the top products in our software categories based on a proprietary blend of user ratings and popularity.

Each Capterra Shortlist may include up to 25 products, but no fewer than 10, depending on the number of top-scoring products in the category.

We use exclusive data and trusted reviews from verified software users to build our Capterra Shortlists, so buyers can confidently build theirs.

The Basics

The Capterra Shortlist methodology scores products on two dimensions: User Ratings and Popularity.

  1. The Ratings component is plotted on the X-axis and considers the overall user ratings of products based on reviews written in the last 24 months. Users rate products on a scale of 1 star (lowest) to 5 stars (highest). 

  2. The Popularity component is plotted on the Y-axis and considers average monthly search volume for a standardized set of keywords for each product, as derived by a proprietary search methodology, as well as the position of the vendor’s domain on the search engine results page (SERP) for each keyword. This is a measure of user interest and relevance used as a proxy for popularity. 

Each score (Ratings and Popularity) is weighted and scaled to a value between 1-50. 

Data sources include approved user reviews, public data sources, and data from technology vendors. Please refer to the Capterra Community Guidelines for more information.

Our Capterra Shortlist reports represent the best and most comprehensive view of products’ recent popularity and ratings, using data from a defined timeframe. Capterra Shortlist reports should therefore be used along with other detailed and real-time information available on each product’s profile page for the most up-to-date view of the data.

Inclusion Criteria

To be eligible for inclusion in a category, products must meet the following criteria:

  • Product has at least 20 unique product reviews published on Capterra within 24 months of the start of the research process for a given report. Two years of reviews provides a sample that we have determined is large enough and recent enough to be valuable to buyers. The criteria is also set  low enough to ensure emerging vendors can be represented.

  • Product shows evidence of offering required functionality as demonstrated by publicly-available sources, such as the vendor’s website.

  • Product serves North American users, as demonstrated by product reviews submitted from that region.

  • Product is  relevant to software buyers across industries or sectors—in other words, no "niche" solutions that cater exclusively to one specific type of user, as determined by our analysis of user reviews and/or market research.

  • Product achieves a minimum normalized overall user-review rating, after application of the Capterra Shortlist method for normalizing and weighting review recency (see description under "Scoring" section).

  • Product achieves a minimum popularity score.

Scoring

To arrive at a final Rating score per product:

  • Scores are calculated for the Ratings dimension based on overall average user ratings. 

  • Average product ratings are normalized for recency and volume of reviews.

  • Scores for each dimension are scaled to a maximum of 50.

Products are represented, included, and/or scored solely based on user ratings and popularity data and independent of any relationship that Capterra has with vendors.

Information for Vendors About The Capterra Shortlist

External Usage Guidelines

Providers must abide by The Capterra Shortlist External Usage Guidelines when referencing Capterra Shortlist content. Except in digital media with character limitations, the following disclaimer MUST appear with any/all Capterra Shortlist reference(s) and graphic use:

The Capterra Shortlist report constitutes the subjective opinions of individual end-user reviews, ratings, and data applied against a documented methodology; they neither represent the views of, nor constitute an endorsement by, Capterra or its affiliates.

Escalation Guidelines

We take the integrity of our research seriously. If you have questions or concerns about The Capterra Shortlist content, the methodology, or other issues, you may contact the Capterra Shortlist team at methodologies@gartner.com. Please provide as much detail as you can, so we can understand the issues, review your concerns and take appropriate action.

Corrections

In the event we discover a data error or omission in a published Capterra Shortlist report, we will publish a correction. See our Corrections Page for comprehensive details.

Tentative Publication Schedule

Please visit our full publication schedule here. Note: This schedule is intended to be used as a guide only and is subject to change at any time at our discretion.

N.B. In January 2020, Capterra Top 20 rebranded to Capterra Shortlist.

Capterra's Top 20 Report Methodology

Methodology: May 2020 - December 2020

The Basics

The Top 20 methodology scores products on two dimensions: Reviews and Web Search Interest.

  1. The Reviews component considers the "overall" user rating (1-5 stars) of products based on reviews left in the last 24 months.

  2. The Web Search Interest component considers average monthly search volume for a standardized set of keywords for each product, as derived by a proprietary search methodology, as well as the position of the vendor’s domain on the search engine results page (SERP) for each keyword.

Each score is weighted and scaled to a value between 1-50. The two scores are combined into a total score (out of a total possible 100 points), which dictates the rank order of products in the report.

Data sources include approved user reviews, public data sources and data from technology vendors. Please refer to the Capterra Community Guidelines for more information.

Our Top 20 reports use a snapshot of review data from a defined time frame and are not updated after publication. Top 20 reports should therefore be used along with the detailed and current information available on each product’s profile page for the most up-to-date view of the data.

Inclusion Criteria

To be eligible for inclusion in a category, products must meet the following criteria:

  • Product has at least 20 unique product reviews published on Capterra within 24 months of the start of the research process for a given report. Two years of reviews provides a sample that we have determined is large enough and recent enough to be valuable to buyers. The criteria is also set low enough to ensure emerging vendors can be represented.

  • Product achieves a minimum normalized overall user-review rating, after application of the Top 20 method for normalizing and weighting review recency (see description under "Scoring" section).

  • Product achieves a minimum web search interest score.

  • Product shows evidence of offering required functionality as demonstrated by publicly-available sources, such as the vendor’s website.

  • Product serves North American users, as demonstrated by product reviews submitted from that region.

  • Product is relevant to software buyers across industries or sectors—in other words, no "niche" solutions that cater exclusively to one specific type of user, as determined by our analysis of user reviews and/or market research.

Information for Vendors About Top 20

External Usage Guidelines

Providers must abide by Capterra’s Top 20 External Usage Guidelines when referencing Top 20 content. Except in digital media with character limitations, the following disclaimer MUST appear with any/all Top 20 reference(s) and graphic use:

The Capterra Top 20 report constitutes the subjective opinions of individual end-user reviews, ratings, and data applied against a documented methodology; they neither represent the views of, nor constitute an endorsement by, Capterra or its affiliates.

Escalation Guidelines

We take the integrity of our research seriously. If you have questions or concerns about Top 20 content, the methodology, or other issues, you may contact the Top 20 team at methodologies@gartner.com. Please provide as much detail as you can, so we can understand the issues, review your concerns and take appropriate action.

FAQs

What are the specific keywords used for the Capterra Shortlist Popularity score? Can software and SaaS providers do anything to improve their scores?

The Popularity score is a proprietary approach to capturing key characteristics of end-user search behavior in a way that is not affected by a provider's SEO prowess. We do not share the specific keywords included, nor changes we make over time. To that end, there are no specific actions for providers to take to improve scores.

We can say that our approach currently takes into account average monthly search volume for a standard set of product-focused keywords, as well as data on average Google search engine results page (SERP) position for the provider’s domain.

Do software providers need to notify you if they’d like their product(s) to be considered?

No, we will automatically consider all products with profiles on our site and that meet our eligibility criteria in a given category.

Do you need any additional information from providers to create the reports?

No, at this time we don’t need nor accept any information directly from our software and SaaS provider community. We obtain all the information we need via our internal databases and publicly available data.

Does it matter which of the Gartner Digital Markets sites — Capterra, GetApp or Software Advice — reviews are published on?

No, reviews are shared across all three sites and all are considered equally in our research.

When will you publish [x] category next?

Please refer to our research calendar for a tentative schedule of reports by month and category. Note: This schedule is intended to be used as a guide only and is subject to change at any time at our discretion.

Are software providers running paid campaigns on Capterra more likely to place higher in reports?

No. A software or SaaS provider's status as a client or non-client is not a factor in any way.

How can providers improve their products’ placement on reports?

While software and SaaS providers cannot directly influence their position in reports, the best way to ensure a product is considered is to collect more reviews.

Can you update a product’s placement in a recent report if that product recently received a number of new reviews?

Currently, our reports are published snapshots in time as denoted by the specific publish date on the report. We do not make changes after they have been published unless there is a significant error in its production. Any such products and new reviews will be considered when we next update the report. Please refer to our research calendar for our tentative schedule of reports by month and category.

How can I compare a current Capterra Shortlist to a previous version?

Capterra Shortlist versions should not be compared because the market definitions and methodology may change over time. Products may also appear under different names from year to year due to mergers, acquisitions and rebranding. Please note, previous versions of this report are considered “archived” and therefore are not updated, but are accurate as of the date of publication.


Was this article helpful?


About the Author

Capterra profile picture

Capterra is the #1 destination for organizations to find the right software and services, so they can save time, increase productivity and accelerate growth.

visitor tracking pixel