Our review process varies slightly to cover adequately the peculiarities of each software category, but generally, these are the steps we go through.
1. We study the technology
Before we dive into a new type of software, we study the underlying technology. We read about it, use it, try to break it, and then fix it. Yes, we are suckers for technology, but the reasons for studying the stuff we want to review go deeper than nerdgasms.
Proper understanding allows us to determine the key points our reviews must focus on. Knowledge of the technology that makes the software run helps us dissect it better, probe the support with more complex queries, and devise diverse user scenarios for testing.
2. We study the company
Typically, once we’ve spent enough time on research and preliminary testing, we venture forth to examine consumer applications before enterprise software, but we are happy to branch into the enterprise offerings as well.
We research which products are at the top at the time and focus our efforts on them but also on the most promising emerging solutions.
Our research begins before the actual software testing. We dig up the dirt on any software company. If it had any security issues or anything alarming from older products, we note it down in our reviews.
Transparent terms of service and fast resolution of bugs and previous mistakes are crucial for a good reputation. Any lack of transparency, worrisome wording in terms of service, or unspecified security measures are all analyzed in our reviews.
Once we have a clear picture about the reputation of the software and the company that develops it, we move on to the actual testing.
3. We purchase the product
Once the reputation is checked, we purchase the software. Typically we purchase the most common or most advertised paid plan.
When available, we check both the free and premium versions of the software to find out what overlaps they have, and what unique features are offered in the premium version.
We pay close attention to:
- Payment options – We go through the payment options thoroughly to make sure the software is accessible for anyone.
- Hidden fees or useless features – We test and analyze the features and deals available. Making sure that the features offered in premium versions aren’t just dressed up free versions. As well as checking that they aren’t missing any crucial key features that the other password managers have.
4. Basic testing
Starting on one device, we test the ease of use and adaptability of the software. Evaluating the UI and how quickly it gets up and running. We start with simple functionality tests, the main features follow, and then we move towards specific offerings.
5. In-depth testing
This is where we take a deeper dive into the features offered. For example, say we are testing a VPN, we would test a service from several locations to get the full picture. We’d use several tools to test the security of the service, set it up on a router, if possible, check whether Netflix and torrenting work as advertised, and so on.
We compile all empirical data for our tested software into a database so that we can create a table ranking each product of that category.
6. Customer Support and Knowledge Base
We check the Q&A’s and forums (if applicable) and reach out to the customer service before using the software. This allows us to get an idea of the support available and the company’s attention to its customers.
Then we follow up with several technical questions, ranging from fairly simple ones to more obscure queries. The idea is to check response times, communication skills, and technical expertise.
Once each of the above is completed, we compile the collected data and begin our evaluation. We give ratings to each part that contribute to the overall rating.
Use – 25%
This one is simple. How easy to use is the tested software? Is the UI too much to handle? Can you seamlessly transition to it and continually use it without problems?
Support – 25%
If the software is straightforward and doesn’t run into any issues, support may not even need to be called. Still, a quick, courteous and knowledgeable technical support and good online resources for self-help are massive benefits
Price – 25%
How much bang for your buck are you getting? What justifies a certain product’s higher price vs. its competitors? How transparent and consistent are the features throughout the plans? All of this and more is weighed into our price evaluation.
Features – 25%
What else does the product offer? The more functioning and useful features, the better.
These 4 factors are what make up the final ratings you see in our reviews. Each factor is rated 1-10, their final average is what makes up the final rating of the specific product under evaluation.
Once a reviewed software accrues a certain number of user reviews, we add their value to the final grade. In the end, user reviews contribute 20% to the overall rating. Then, each of the aforementioned categories contributes 20% instead of 25%..
You can see our software reviews with or without factoring in user ratings, as well as sort by features or according to what you think is important in your software.