4 Signs Your Business Intelligence Software Proof Of Concept Is a Dud

So you've narrowed the business intelligence software vendors you're considering down to a shortlist. You know what your business needs out of the software, and you have a good idea of where to get it.

This is a great start, but what comes next is equally as important: the proof of concept. When you're buying software, a proof of concept is like taking a test drive when buying a car. In both cases, you're looking at how the product handles in settings you'll encounter on a daily basis.

If you're shopping for a minivan or an SUV, you want to make sure it'll fit your kids, a few friends, and their soccer equipment before you sign.

If you're shopping for business intelligence software, you want to make sure it can integrate your data sources before you commit.

Much like the telltale signs of a bad test drive (squealing brakes, a smoking engine, or the salesman sticking his feet through the floor, Flintstones-style, to get the car moving), there are sure signs of a bad software proof of concept you need to keep an eye out for.

Here are four signs your business intelligence software proof of concept is a dud. If you see any of these, it may be time to consider a different vendor.

1. You have to pay for your proof of concept

Our first red flag is a vendor asking you to pay for the proof of concept. You'd be suspicious if a dealership charged you to drive a car for 30 minutes, right? You should be equally wary of vendors that want you to pay to see how well their product integrates your data.

Buying a business intelligence solution isn't a one-and-done purchase. It's a relationship with the vendor and the community that uses their program (think customer support, vendor education courses, and online forums). Therefore, your vendor should view you as a partner, rather than just a source of income.

A trial run like a proof of concept isn't a transaction, but a chance to feel each other out and see if the basics of the relationship are in place.

A good proof of concept bodes well for later success

Preparing for a proof of concept requires work from both parties. The vendor has to build a basic version of the solution they'll design for you, while you need to know what you want out of the solution (more on this later). By the point of the proof of concept, the savvy business intelligence shopper has already determined exactly what they need.

2. The vendor isn't using your data

The best way to make sure the BI program you're shopping for can handle your data is to see it handle your data. Test driving a car in Florida doesn't make much sense if you need to see how it handles in the snowy land of Maine where you live.

Make sure the dashboards use your data.

Capterra senior software engineer Venka Ashtakala notes that "greater confidence" in the solution is a major reason for using your own data in the POC. Your data "may have some edge cases or other data peculiarities that test data may not have."

The data you're looking to feed into your business intelligence program may be designed, or customized, in a unique way that requires different treatment. "At a higher level," Ashtakala says, "using 'real' data guarantees that any business logic or rules that are baked into data will be handled properly by the proof of concept code being developed."

A proof of concept with your own data provides the same sort of thoroughness you'll want when using the program on a regular basis.

3. The proof of concept doesn't demonstrate the vendor's claims

Another thing to keep in mind during your proof of concept test is the extent to which the vendor does—or doesn't—live up to their claims, whether those claims are unique to the POC or are more general, branding claims.

If a vendor claims a fast product, for instance, then it shouldn't work at the speed of a hamster languishing on a rusty wheel.

If you want to work on a hamster wheel, however, that's another matter

The reasoning behind this one is self-explanatory: if the vendor can't deliver on what their brand promises in the POC, it's unlikely they'll be able to deliver later on.

Even if the proof of concept meets your specific requirements, a product that advertises itself as easy-to-use shouldn't be confusing. It suggests an inconsistent brand, which further suggests future problems in your longstanding relationship with the vendor.

4. The proof of concept doesn't meet your needs

While the first three red flags in this piece focus on the vendor, this fourth tip swings the spotlight back on you and your business. Once you've identified your needs, it's important to evaluate the proof of concept to ensure it meets those needs.

Ensuring that your vendor meets your needs is important enough that Jason Remillard of Data443 calls it "the main indicator of a failed POC."

How do you know you're dealing with a successful proof of concept? Remillard says that the vendor will have "confirmed customer requirements, documented them, re-validated them with the customer," and gotten the customer to sign off on each requirement. This level of due diligence not only demonstrates that the vendor knows your needs, it "ensures everyone is focused on the right efforts," and puts you in the right position to manage your relationship with the company that will aid your journey to becoming data-driven.

This level of vendor thoroughness makes for a successful proof of concept and sets you up for successful habits once you're regularly using the program. With well-defined and agreed requirements, both parties have "something to reference," which reduces possible confusion. According to Remillard, a document of established goals also streamlines the process and makes it amenable to change control.