How We Evaluate Security and Antivirus Software

Security and Antivirus Software

Which security and antivirus products work best to keep you and your computer safe? We put them to the test to find out.

How can you tell whether an antivirus or security suite program is working properly? It differs from a motion-detecting camera. You can’t shake it and see it work. We put their claims to the test in a number of ways to see if they’re working properly.

Some packages include parental control, but others omit this feature. Some antivirus programs that purport to be standalone also include firewall security.

Security and Antivirus Testing Real-Time

The only way to verify that these security and antivirus measures are effective is to assault them with actual malware. We use virtual machines for this testing so there is no risk of missed infections.

Every spring, we collect a fresh set of malware samples from security vendors. We download thousands of samples, and whittle them down to a manageable number.

We’re looking for samples that make changes to the file system and registry. We analyze each sample using a variety of hand-coded tools. Some models detect and stay away from malicious activity while running on a virtual machine. We simply don’t use them; instead, we record what texture changes each sample makes.

Security and Antivirus Sampling Checking

Norton’s security suite lets you see what models the security and antivirus removes from view. Some products have real-time protection that starts immediately and destroys known malware. You can download samples from cloud storage or do whatever is necessary to start real time protection.

Next, we launch each remaining model and see how the antivirus handles it. We record the total percentage detected, regardless of when the detection occurred.

Malware detection is essential but insufficient; antivirus must actively stop the attack. A small application scans the system to see whether any registry modifications or file installations have been made by the infection.

A product that stops the malware sample from installing any executable traces at all is worth 10 points. Detecting malware, but failing to prevent installation of executable components earns half credit, 5 points. If one or more malware processes running despite the anti virus’s protection efforts, it is worth just 3 points.

Web-Level Protection Testing

The best time to destroy malware is once it reaches your computer. Many security and antivirus products integrate with your browsers and steer clear of known malware-hosting pages. If security doesn’t start at that level, there’s always a chance to destroy the malware payload during or immediately after download.

While our basic malware security testing uses the same samples per season, the malware hosting URLs we use to test web-based security are different each time. We get those links from a feed of very fresh malicious URLs found by London-based MRG-Effitas and use URLs that are no more than a few days old.

How do you tell if an antivirus is blocking access to a URL, killing the download, or just sitting there doing nothing? Using a small built application, we go down the list starting with each URL. We reject anything that doesn’t point to a malware download and gives error messages.

The score in this test is the percentage of URLs that the antivirus prevented from downloading malware, either by completely cutting off access to the URL or by deleting the downloaded file. Perfect scores of 100% are not unusual, and the majority of evaluated antivirus programs provide 90% or superior security.

Phishing Detection Test

These phishing websites mimic banks and other major sites. If you enter your login credentials, you’ve given away the keys to the kingdom. Phishing is platform-specific; It works on any operating system that supports web browsing – it doesn’t matter what operating system.

The best security programs detect these new fakes using real-time analytics and blacklist them quickly. People who rely on simple-minded blacklisting generally get lower scores.

We look at whether Chrome, Edge and Firefox protect against phishing attacks that try to steal passwords and usernames. We also examine whether each product detects fraud when a website tries to impersonate another site or capture username and password data from a protected machine.

As with web-level security testing, scores vary widely. Some products achieve 100% detection, while others can’t even beat protection across three browsers.

Tests Spam Filtering

The average consumer has little to no need for e-mail spam filtering, an Austrian test lab has found. Microsoft Outlook alone blocked nearly 90 percent of spam, and most packages performed well. One of the founders of AV-Comparatives claimed that hosted mail solutions like Gmail eliminate the need for a spam filter.

As part of our spam testing, we download thousands of messages and manually check to see if any spam has slipped into the inbox or, worse, that valid mail has been marked as spam. This test took more time and effort than our other tests. There is no point spending maximum effort on an aspect of lesser importance.

Can it be used with an unsupported client? Is it limited to POP3 email accounts or does it handle IMAP, Exchange or web-based email? Currently, we carefully consider each package’s antispam capabilities, but we are long-suffering from downloading and analyzing thousands of emails. Which email clients does it support?

Tests Security Suite performance

Security and Antivirus suites are designed to take up precious CPU and other resources to do their job. Security companies have designed their products so you won’t ever see significant slowdowns.

After a reboot, Windows starts asking Windows to report CPU usage levels once per second. After 10 seconds in a row, CPU utilization is no more than 5 percent, which declares the system ready to use. We run several repetitions of this test and compare the mean of the repetitions when there is no aggregation.

Role of Security Suite

You shouldn’t reboot your PC every single time you use it. A security suite that slows down daily file operations can cause more harm in your operations. To check for that kind of slowdown, we spend time running a script that moves and copies large files between hard drives.

There are still some packages that significantly slow down one or more of these tests, but their number is dwindling. At the other end of the spectrum, we’ve found many cases where tests ran faster after installing the package.

Tests Firewall Security

A typical personal firewall protects the computer from external attack. To test the security, we use a computer connecting through the router’s DMZ port. This gives the effect of a computer directly connected to the Internet. The built-in Windows firewall handles all port spoofing, so this test is only a baseline.

Programmatic control was painfully interactive on early personal firewalls. Every time an unknown program tries to access the network, the firewall prompts the user to allow or block access. This approach is not very useful because the user usually does not know what action is correct.

It is becoming more common for suites to provide this level of granular programmatic control by default. For those products, we run it before testing, and use an unknown application to verify its functionality.

Best Firewalls

Since we don’t write malware, this isn’t something we can test, but we can observe this functionality during malware security testing. The best firewalls automatically configure network permissions for known good programs and remove known bad programs.

Naturally, the manufacturer of the exploited product releases a security patch sooner rather than later, but until you apply that patch, you remain vulnerable. They create exploits to compromise system security using any vulnerabilities they find.

Intelligent Firewalls

Intelligent firewalls intercept these exploit attacks at the network level, so they never even reach your computer. Using the CORE Impact (opens in new window) penetration tool, we hit each test method with about 30 recent exploits and record how well the security product blocks them.

We also examine whether it is possible to use third-party tools to halt or stop the product’s crucial Windows services. Finally, we run a sanity test to see if we can easily disable the malware encoder security protection.

Tests Parental Control

A typical parental control program protects children from inappropriate websites and tracks their online activity. Other features include limiting chat conversation and monitoring Facebook postings for potentially hazardous content.

We always perform a sanity check to ensure that the content filter is properly blocking inappropriate websites. Finding porn sites to test is a snap. Any URL made up of a size adjective and the name of a casually covered body part is already a porn site. The majority of goods pass this test.

Filtering Of Content

Here’s a three-word network command that disables some simple-minded content filters. We check if we can defeat the filter by using a secure anonymous proxy website.

The best products don’t rely on a computer clock to keep track of their date-and-time. We check if the time scheduling feature is working and try to prevent it by resetting the system date and time.

After that, we try to disable it by moving, copying, or renaming the application if it claims to restrict the use of specific programs. We add make-believe words, like “fnord,” to the blocklist and check to see if it hasn’t already been transmitted if it promises to remove offensive words from emails or instant messages.

Analyzing Antivirus Lab Test Results

We don’t have the resources to conduct thorough antivirus tests performed by independent labs around the world, so we rely on their findings. We follow four laboratories that regularly publish scored test results, and use their results to help inform our reviews.

AV-Test Institute regularly tests antivirus programs through a variety of tests. Best products score a perfect 18 points on this test. To achieve certification, a product must score a total of 10 points and not have zero scores in any category.

Products are credited for preventing infection at any level, such as blocking access to a malware-hosted URL or detecting malware using signatures. Best products often achieve 100 percent success in this test.

Importance of Performance

AV-Test’s researchers measured the difference in time required to perform a dozen common computer tasks with and without an antivirus security product. By averaging multiple runs, they can identify how much impact each product has.

antivirus software scans over 600,000 legitimate files to make sure they don’t identify them as malware. Usability testing measures whether an antivirus program incorrectly flags a legitimate program or website as malicious. Researchers install and run a collection of popular programs to spot any unusual behavior by antivirus.

AV-Comparatives, based in Austria, tests safety equipment that passes a series of tests to earn an Advanced or Advanced+ certification. If a program passes, it earns an Advanced certification; if not, it is simply referred to as tested. The tests are run by AV-Test, which works closely with the University of Innsbruck.

The Lab’s Test

This lab’s file detection test is a simple, standard test that checks about 100,000 malware samples for each antivirus. It measures any impact on system performance.

SE Labs (opens in new window) typically includes no more than 10 products in its testing. Researchers capture real-world malware-hosting websites and use a replay technique so that each product experiences the same drive-by download or other web-based attack. It is very realistic, but difficult.

A program that completely blocks one of these attacks scores three points. If it stops attacking without a full cleanup, it gets one more point. Some products (especially earlier versions of Windows Defender) have scores below zero.

Evaluation of Researchers

They assess how well each product avoids falsely identifying malicious software. They combine the results of these two tests to certify successful products at one of five levels: AAA, AA, A, B and C.

This test simulates real-world protection against current malware. A product that completely prevents any contamination through sample collection receives Level 1 certification. Any product that does not achieve one of these levels will fail.

Our total lab results chart reports the average of these scores and the number of lab tests. The best result is a perfect score of 10 based on results from all four labs. If only one laboratory has tested a product, we consider it insufficient information for a total score.

Testing a VPN is very different from testing any other part of the security suite, so we’ve provided a separate explanation of how we test VPN services.


How does antivirus software provide security?

Security and Antivirus software stops malware from harming your device by identifying, containing, and/or destroying dangerous code. Modern antivirus programs automatically update themselves to offer protection against the most recent viruses and malware.

Which kind of antivirus you should use?

Because its subscriptions provide security for every device in the home at an affordable price, McAfee Antivirus Plus is our top pick for multiple devices.

How do I check for viruses?

Toggle Play Protect on in the menu by opening the Play Store app, tapping your icon or avatar in the top right corner. Install malware protection software. The best way to automatically find and remove malware from your Android phone while guarding against new infections is with an antivirus app.

Leave a Reply