Numbers and Opinions? The Sweet Point of Quantitative Versus Qualitative Analysis
What’s the next best thing that product managers love the most after great leads? It’s data. While other departments might be scared of collecting and handling data, great product managers know that armed with the right data, they have the world at their fingertips.
However, data on its own is nothing without analysis.
There are two main ways that product managers analyze data. There is the quantitative method, which involves statistical analysis and formulas, and the qualitative method, which relies on personal opinions.
So, which one is better for product marketing purposes? The answer may not be what you think! In this piece, we will discuss how each of these methods can provide useful product and marketing insights that you can learn and make money from.
Numbers, or quantitative analysis
When we think of the word “data”, most of us think of Excel, spreadsheets (e.g., for accountancy with Excel) and racking our brains trying to figure out what is what. Like it or not, one of the biggest benefits of data is that it’s easy to measure, i.e. quantify.
source: giphy
What is quantitative analysis?
Quantitative analysis is the process of collecting, organizing, and presenting data in numerical form. This method involves statistical formulas to draw conclusions from that data. For example, describing that a landing page is underperforming by showing its poor conversion rate of 1.2%.
In other words, you run tests and research to find the specific results that can be measured in cold, hard numbers.
Why use quantitative analysis? Statistics and numbers are used for two main reasons.
- to understand existing business trends or problems and predict future ones.
- to solve questions when it’s too difficult to get an answer by interviewing customers
In the first case, the intent is clear. In the second case, it’s simply easier to use numbers and interpret them rather than talking to lots of customers one on one to determine what the truth is about their specific situation.
Benefits of quantitative analysis
Sure, having just numbers on paper might sound like a poor starting point for a product campaign. However, there are some upsides too.
You can reach a higher sample size with quantitative research
While it’s rather easy to talk with a handful of customers about their experiences and overall customer satisfaction, trying to understand what thousands or millions of users think takes significant time and resources. Quantitative analysis is the solution to this problem since data can be collected from a larger group in less time.
For example, some companies even look at product reviews on Amazon when they are considering adding new products to their lineup because that platform has over 300 million active buyers who leave feedback on everything they purchase! By looking through these reviews and seeing how customers responded (positively or negatively) you have access to valuable information without having to interview every single person yourself.
Better and quicker business decisions
The second advantage of using quantitative analysis is its ability to help make business decisions by drawing conclusions based on data from past business events. While it is unnecessary to predict every possible outcome, a company can make decisions based on previous behavior and adjust its product accordingly.
Samples are randomized so it excludes bias
The third advantage of quantitative analysis is that it removes bias. When your team asks customers for feedback, they may get a biased opinion which can lead to incorrect business decisions being made. This can happen with quantitative data too, but much less so compared to qualitative analysis.
The downsides of quantitative analysis
The quantitative approach has its own drawbacks as well. First and foremost, there is a lack of detail. As you get to see the numbers behind the (lack of) performance of a certain page, you don’t get to see what caused these numbers. This is crucial in a situation when something goes wrong, but it’s just as important when you want to figure out why a certain page shot up in conversions as well.
Another problem is that you’re looking at either an isolated situation (such as one page that is underperforming) or your entire website. Quantitative analytics rarely give you the insights you need for a specific user journey.
Opinions, or qualitative analysis
On the other end of the data analysis spectrum, you have qualitative analysis or dealing with opinions. Instead of collecting raw data in the form of numbers, you take opinions from real people. While it’s not always the most accurate and fastest way of getting data, it has its own advantages.
source: giphy
Data can only tell you as much as you’re capable of interpreting it. In other words, a 1.2% conversion rate could mean the world to a superb marketer and they’d instantly know what to do with this number. To someone in accounting, it would be like trying to teach a chicken how to wrestle.
Note that qualitative analysis is not the same as sentiment analysis, where you only aim to understand how a customer feels about your product/service. The qualitative approaches aim to collect descriptive data, i.e. from a review website or online forum, which is also known as qualitative content analysis.
What is qualitative analysis?
While quantitative analysis is all about what happened, qualitative analysis tells you why and how something happened and allows you to get real insights from customers. For example, you may get 1,000 websites visits today, which is low compared to your usual of 5,000 - so something is definitely not right. The quantitive analysis is the fact that you have 1,000 visits and that’s it.
On the other hand, qualitative data analysis will look into the reasons why this happened. For example, a piece of your code got broken, your exit-intent popup stopped working, an image used on the page is broken or something else. In short, it’s a description of what happened and what may have caused it. Qualitative content analysis means talking to customers to find out what took place and then documenting their accounts.
Qualitative analysis works when numbers alone can’t tell the full story.
The benefits of quantitative analysis
One of the biggest benefits of qualitative analysis process is that it gives depth to your research. Where numbers fail, the stories shine through. Instead of wondering why a certain page performed in a certain way, you get to see the actual story behind it. You don’t have a number that you have to accept at face value. Instead of reporting to your manager that your bounce rate is 80%, you can show them the specific chain of events that led to this unfortunate situation.
It’s also more flexible as it allows multiple interpretations of a single data point. If we take those 1,000 visits as an example, there could be countless reasons for this result and qualitative research helps spark a discussion rather than focusing on numbers alone.
That’s the last highlight we’d like to stress too - qualitative means that you can discuss certain results instead of accepting them as they are, without looking into more detail.
The downsides of qualitative analysis
This is not to say that qualitative research doesn’t have its own cons. First of all, the sample size of a typical qualitative study is smaller. Since you are looking into a specific case (or a handful of them), you’re looking at a fairly small sample and you can’t make any major conclusions based on a single case.
The second major drawback is that you need an expert set of eyes for this approach to analysis. In order to get tangible feedback from analysis, you really need the opinion of an expert and not just anyone willing to give you their two cents.
Third, it’s difficult to generalize. Since you’re zooming in on one case, you can’t make any generalizations that apply to your entire customer base.
Last but not least is anonymity. If it’s important for your researcher/target audience to remain anonymous, it’s hardly possible in a qualitative research scenario, since they have to provide their own personal opinion.
Mixed methods analysis – finding the sweet point
While both methods have their benefits, we think there is a certain “sweet spot” where qualitative and quantitative research intersect. By using data from past events as well as customer interviews (the ‘soft’ facts), companies can make better business decisions in less time without any unnecessary biases or assumptions getting involved!
The best part about the mixed methods study? Companies don’t need to choose one method over another - by utilizing them together you will be able to provide insight into all areas of your company’s operations.
What it’s all about
A mixed-methods analysis is at the cross-section of quantitative and qualitative research. Instead of looking at just data or just giving an opinion on a situation, it combines both to give you a better idea of what happened with your website, product, app, or any specific page out there.
Website session recording
Let’s say that you have a page that performs really well and converts at 5%. Overnight, the conversion rate starts plummeting and you’re well below your average at a 2% conversion rate. The number of visits stays the same but the conversions are not coming in.
Session recording is the ideal tool for a mixed-method analysis. It gives you real-time insights into what the users are doing and what the visitors are doing. You can see them come to a page and take specific actions which result in the lack of conversions.
What’s even better is that apps like LiveSession let you segment specific audiences. For example, you can look at mobile visitors only if you see that this audience is the one causing issues. Or you can single out specific demographics, such as users based in the USA.
The visitor journey
Any good marketer knows that a visitor doesn’t just land on one page out of the blue and convert immediately. The sales funnel always comprises at least a handful of pages. If something seems to be off, you can look into the entire journey to find out which step made them take a wrong turn.
Thanks to LiveSession’s user tracking, you can identify a specific customer and see the steps that they went through before making a purchase or failing to make one.
The heatmaps
Sometimes, you need a birds’ eye view of things. Heatmaps are one of those qualitative approaches that can show you the why behind your most troubling page in just a single image. In a nutshell, it’s a graphic representation of how users engage with your website/tool. The “hotter” the color of the map, the more engagement it has with your visitors.
Heatmaps are an effective way of connecting the dots.
Custom metrics and dashboards
We want to enhance LiveSession in the near future with more features to collect valuable quantitative data. Therefore, we are currently working on Custom Metrics and Dashboards, which will be released soon.
Match your method to the problem
Simply put, you cannot state that either qualitative or quantitative analysis is the better choice for any project out there. As both quantitative and qualitative approaches have their strengths and weaknesses, it’s a far better idea to look into the problem that you’re trying to solve and then decide on the approach you want to take.
source: myperfectwords.com
For example, you run an eCommerce store and you want to tie your average visit length to your conversion rate or see how many pages someone visits before making a purchase. In this case, quantitative analysis is your best choice because you’ll have all the data within a few clicks instead of scouring review sites, for example.
On the other hand, you may have a situation where a landing page is performing okay but it’s not getting the right conversions within your desired demographic - dog owners on a website that sells pet accessories. You can do a qualitative thematic analysis to see if the copy, the design, or the overall user experience is not meeting their expectations.
And finally, say that you have a SaaS website that constantly gets a high amount of traffic and visitors click through so the bounce rate is actually good. However, very few of those visits actually convert. This is where a mixed method is the best approach because you’ll get to see the data and the story behind the data in one go.
Wrapping up
The information that quantitative research provides may not be as reliable or in-depth as qualitative analysis, but it has its benefits! So what’s the sweet spot between these two methods? The answer lies within combining them both for deeper insight into your customer base.
The best way to go about it is to use an app that analyzes the situation for you! LiveSession takes a deep dive into your website, app, or page to show you not only what happened, but why it happened. If you’re looking for one solution that will give you all the insights you need without spending thousands on a UX specialist, sign up for LiveSession today!