Anti-Malware , Endpoint Security , Technology
Anti-Virus Wars: Sophos vs. Cylance Sophos Says Product Duel Was Rigged; Cylance Says No
The market for endpoint protection products is very large pie: Market researcher IDC estimates the consumer and enterprise market is worth just over $9 billion, combined, and will grow around 3.9 percent annually over the next three years. It's always been a fiercely competitive market, and vendors compete in no-holds-barred matches for customers.
See Also: Creating a User-Centric Authentication and Identity Platform for the Healthcare Industry
The latest conflict has erupted between Sophos and Cylance, two fairly well-known vendors. Both companies turned down repeated interview requests from Information Security Media Group for more details about their dispute, which unfortunately makes either side's position impossible to verify. But even so, the standoff shows how vendor competitiveness can quickly turn ugly.
In a blog post on June 29, Dan Schiappa, senior vice president at Sophos, accused Cylance of purposely hobbling Sophos' product during a one-on-one malware duel at a recent security event in Las Vegas.
Cylance does a road show called The Unbelievable Tour where it demos its product, which doesn't use traditional anti-virus signatures. Instead, Cylance's Protect product uses an algorithm to detect abnormal activity. Cylance is one of a range of vendors, including SentinelOne, that use this approach, which they claim offers better protection against malware.
Default Dispute
Schiappa writes that a customer from Chicago asked to see the default settings for Sophos' product during the test.
"On reviewing the settings, the customer discovered that key (and default) protection settings had been disabled," Schiappa writes. "When the customer insisted that Cylance enable the proper default configuration and re-run the test, Sophos beat Cylance."
Sophos also ran a test again after acquiring Cylance's product from a reseller, Schiappa writes. Sophos didn't cherry pick malware samples or alter the default, vendor-recommended settings. The video was then posted on YouTube, he writes.
Schiappa then alleges that Cylance contacted the reseller who provided the company's software to Sophos and threatened retribution if the video was not withdrawn. Sophos caved and removed the video.
"Again, to be very clear: the only reason we elected to take the video down was because the reseller was concerned about threats and pressure from Cylance, not because we believed the video was somehow inaccurate," Schiappa writes.
Sophos' version of events couldn't be verified with Cylance. But the company briefly responded to Sophos in a blog post of its own on June 30 dismissively titled "Sophos, So Far."
"This conversation has gone on long enough and wastes everyone's time," writes Ryan Permeh, Cylance's chief scientist and founder. "We strongly urge customers to test any solutions on their own systems and networks. It is the only truly independent and 'real world' metric that ever matters."
Testing is Tough
Anti-virus software testing has always been contentious. Vendors have often taken to task independent testing organizations such as AV-Test.org and AV-Comparatives.org, quibbling over malware samples used in tests and questioning methodologies, especially when products fare poorly.
Independent testers have a tough task, particularly as security products have evolved to incorporate more behavioral analysis as well as signature-based approaches. When it comes down to it, vendors are unlikely to promote any test result that isn't favorable.
Vendors also won't publish unbiased tests, says Simon Edwards, founder of the independent security software testing company SE Labs, which is based in London. Rather than unverified demonstrations, third-party tests are the important ones since the results are the unvarnished truth.
"This is a commercial reality, and they [vendors] will always claim to be the best, or one of the best, even if the technical truth is different," Edwards says. "No one test is perfect, but look at a combination of tests from different sources to judge the merits of a product."
'Truly Unbelievable'?
Edwards says Sophos makes a good point in its blog post that Cylance has not participated as widely in independent tests as other vendors.
"[Cylance's] own tests, embodied in its Unbelievable Tour, were truly unbelievable and literally incredible, in as much as they were not credible," Edwards says. "It was a clever marketing idea that achieved good press coverage, but now it's time for the company to expose itself to the same scrutiny that its competitors have had to face for many years - serious, independent and ethical testing."
From a marketing standpoint, meanwhile, this isn't the first case of vendors - and especially newer players - in the IT security market employing guerilla-marketing tactics, and claiming that their technology is superior solely via tests they've conducted on their own, says Andreas Clementi, chairman of AV-Comparatives.
But such claims don't always stand up to third-party scrutiny. For example, AV-Comparatives published a report in February detailing the results of its tests pitting Cylance's Protect product against Symantec's Endpoint Protection. The testing organization had trouble obtaining Cylance's product, as two resellers refused to license it to AV-Comparatives, according to the report. It eventually obtained access to the software through a third party.
The results? Symantec's Endpoint Protection fared far better than Cylance Protect, according to the AV-Comparatives report. It found that Symantec stopped 100 percent of in-the-wild malware and 92 percent of exploits, compared to 92 percent and 63 percent for Cylance.
What Goes Around?
Back to the Sophos versus Cylance spat: Just six days before Sophos went public with its gripe, Cylance published a blog post alleging that it was the victim of another anti-virus vendor meddling with its product before a test.
An unnamed "legacy AV vendor," which Cylance says sees hundreds of millions of dollars in annual revenue, produced a video of a product comparison in cooperation with a partner. Cylance says its product had been obtained by a "rogue employee" of the partner, who then disabled key features before testing.
"It's not surprising that this legacy AV vendor would resort to dirty tactics and essentially use a partner to wage a proxy war," Cylance writes. "The vendor was caught with their hand in the cookie jar and [is] now attempting to spin the matter into something else entirely."
Details of that situation couldn't be verified with Cylance, which didn't make executives available for interviews.