Cisco Study Shows the Basic Flaw in Whitelisting Solutions

Some days you wake up and the world hands you a completely unexpected gift.  This morning I found an article on the SC Magazine site that provided statistics from a Cisco survey about employees and IT security policies.  Some stats from the article:

  • 24% of employees are unaware that IT policies exist.
  • 10% said that IT policies are never communicated.
  • 32% of employees said that the policy was only communicated once per year.
  • 35% of employees that are aware of IT policy said IT does not provide an explanation or rationale for why it exists.
  • 20% of employees make a conscious decision to break IT policy because they believe these policies are not enforced.

These statistics do not paint a picture of a well informed user community.  Users do not know the policies, don’t understand the policies, or don’t understand why there are policies.  The few that seem to understand often choose to willingly ignore them.

The most telling statistic indicated that 40% of the employees break IT policy because “they need restricted programs and applications to get their job done”.  In other words, they know they are breaking policy but make the decision to willingly do so and feel justified because they feel it is critical to their jobs.

So why is this study a gift for me?  I am frequently asked to contrast and compare Triumfant and our capabilities against whitelisting tools.  I have a good answer, and while I normally become extremely animated about the subject and speak in authoritative tones, I did not have hard evidence to fully back up my position.  Until now.

You see, whitelisting sounds really smart and effective in explanation, and are often cited as an alternative to signature based tools and falling malware detection rates.  There are animated claims about its effectiveness aginst zero day attacks, the advanced persistent threat, rootkits, and the cough due to cold.

If you dig deeply past all of the hype, you will find that whitelisting tools work in three modes:

  • Notify mode will notify the appropriate IT staff if a user installs an application not on the white list.
  • Warn mode will notify the user that they are installing an unauthorized application and provide them the option to stop the install or proceed.
  • Block mode will automatically block the installation of any unauthorized application.

These are not my descriptions – they are from the literature and documentation of the whitelist vendors.  They just don’t surface in the sales presentations.

The documentation clearly states that block mode is only available if the environment is locked down.  For those environments that have even small degrees of flexibility and some personal use capabilities, whitelist solutions only work in warn mode.  Their words, not mine.

Therefore, the efficacy of the whitelist solution now rests in the hands of the user of the machine.  Yes – the very same users statistically characterized by the Cisco study.  The user who likely made a conscious decision to install the program, has a one in four chance of being completely unaware of IT policies, and, if aware of the policies, either does not understand them or is willing to break them.  Hardly sounds like a recipe for closing gaps in endpoint security.

This is not my first rodeo and I have been dealing with the user community since I helped support a quaint old notion called the “Information Center” back in the early 80’s.  Since then, every shred of evidence and experience tells me that most users presented with a warning screen from the whitelist tool will blithely blow right past it.  Now I have the statistics to back that up.

My contention is that only a small number of organizations are locked down, and therefore implementation of a whitelist tool can only be done in warn mode, therefore putting critical protection decisions into the hands of the general user population.  The population that may not know, may not care, and will likely be perturbed that they get a warning screen.  These statistics clearly indicate that there will be more than a trivial amount of users that will circumvent the protection either through ignorance, apathy or choice.

So excuse me if I do not jump on the “whitelisting will cure all of your problems” bandwagon.  And BTW, the same warning process is employed by the prevalence based technologies such as Symantec Quorum that Symantec and McAfee are touting so highly.  The reliance on the user as part of the protection mechanism is equally flawed.

Triumfant does not rely on the user to make evaluations or give them the option to violate policies.  We enforce configurations and policies on a daily basis, and it is an informed administrator that evaluates potential malicious activity and makes the decision to remediate such problems.

So now I have some statistics to support my animated hand waving. Amazing what a little gift like some statistics will do for your day.

Symantec Says Black Hats are Winning – We Say Don’t Throw in the Towel Yet!

There is an interesting article floating about on multiple web sites with the title: “Black Hats are Winning, Symantec Says”.  The article appears in ComputerWorld, PCWorld, NetworkWorld and other sites.

While this may be an interesting admission by Symantec, I think the bigger problem is that we are allowing the black hats to out-innovate us.  More precisely, we are allowing market dynamics and an aversion to adopting new technologies to stifle innovation unnecessarily and therefore give the adversary an even bigger advantage.  We are, in some sense, helping them win.

Organizations trusted the AV vendors to address the signature problem and got the long list of technologies cited in the article: heuristic, behavioural and intrusion prevention technologies.  The AV vendors trotted each of these technologies out to solve the shortcomings of their solution and each proved in turn to have significant shortcomings.  The cycle perpetuated itself because traditional thinking and the reliance on prior knowledge hampered these supposed solutions.  Because these technologies failed, Symantec is now emphasizing their reputation-based security, while McAfee has been leaning hard on their whitelisting technology.

The very real innovations that are available today often do not get the opportunity to prove their worth and show that they can help win the ongoing fight.  The big vendors will protect their turf by telling customers that they “can do that” when a closer look may prove otherwise.  Much of what the 800-pound gorillas bring to the market is based more on justifying their latest acquisition rather than innovating to keep up with the bad guys.  Organizations respond by taking what they perceive to be a less chancy path and trusting the big vendors in spite of their track record, because innovation often comes from smaller companies that may be perceived as introducing risk due to their size.  This cycle serves to hand the innovation advantage to the adversary.

The adversary already has an advantage, because defense will always trail offense.  What we must collectively avoid throwing in the towel and allowing our actions to widen the gap needlessly.  Organizations must look past the traditional vendors to new and innovative detection technologies, and the larger established 800 pound gorillas in the room must stop stifling innovation through their “not invented here” attitudes.

No matter what Symantec or any other traditional vendor may say, there is no reason to throw in the towel if organizations would think beyond traditional companies and approaches and embrace innovation.  We obviously think Triumfant is one such innovation, but I have seen many other good ideas on the market.  Let’s not declare the battle over just yet, but instead let’s make sure we create an environment where innovation can flourish and be readily engaged in the battle.  The adversary certainly has no such artificial barriers.

RSA Shocker (Not): Symantec Admits Traditional Signature Based Tools are “Not Keeping Up”

“Traditional signature-based approaches to security are not keeping up.  What we’ve had to do is come up with a new approach. The idea is it has to be able to deal with attacks that we’ve never seen.”

Words from some maverick security company?  Hardly.  These are the words of Symantec CEO’s Enrique Salem from his Tuesday RSA Conference keynote.  And he is about to tell the assembled RSA crowd that Symantec’s prevalence technology is the answer to the vexing problem of rapidly emerging and constantly evolving threats.  I can’t fault his message – his company paid handsomely for that keynote spot so he can proclaim his new technology as the 2010 silver bullet.  But in my opinion, Salem and Symantec’s new found honesty regarding the efficacy of AV is late, awkward, and does little to provide real leadership to the market.  The industry leaders should not feel all self congratulatory in finally admitting a problem they have ignored for far too long.

I had a similar experience listening to a CEO in denial say something equally late and awkward before at the 1999 Sapphire Conference (SAP user conference) in Philadelphia.  SAP was acting like the World Wide Web was simply not happening all around them because it was so foreign to their core technology.  In his keynote, then SAP CEO (or COB) Hasso Plattner grudgingly referenced the internet as an “emerging technology” but was still ultimately dismissive.  I remember thinking “sir, I think the internet has already emerged and no dismissal from you can change that fact”.  Actually, I think my exact thought was “Emerging? Dude, internet done emerged!”

What confounds me is that companies still somehow either believe or want to believe that companies like Symantec can solve this problem.   Not one person in a company or government agency that fights what has been called the advanced persistent threat tells me that they believe that prevalence technology is a viable solution for what Salem calls “the attacks that we’ve never seen”.  Same with whitelisting, which is the proposed answer for companies like McAfee and Lumension.

(As a complete aside, one vendor actually touted “intelligent whitelisting” at RSA, I assume implying that somehow intelligence had been left out of previous whitelisting attempts.  I could see people everywhere saying “AH! I was supposed to be intelligent about whitelisting!  Now I get it.”)

I think it is disingenuous for companies that have been at the front of the A/V wave to feign public shock that signatures are no longer viable when their own customers have been pleading with them for years and years to step up and make the jump to newer technology.   We of course have been pointing out the problem for some time, with our Worldwide Malware Signature Counter providing a visual for the problem.  I also think it odd that a company like Symantec would post a reports showing that 100% of the enterprises they polled for a recent study had been attacked (see an interesting view of FUD surveys in John Pescatore’s blog here).  The math is simple: if Symantec represents 40% market share and 100% were attacked, aren’t they saying that they failed to protect 40% of the enterprises represented in the survey? Seriously, am I missing something here?

Let me be clear.  The answers to the problem Salem raises do exist.  You and your organization are simply going to have to look outside of your AV suite vendor to find it.

Symantec Quorum – The Carbon Based Life Form Problem

I am still a bit baffled by the rush to embrace the reputational aspects of products like Symantec Quorum.  I do get how it works, I do get that it adds value and can help a user see if the application they are loading may be malicious based on its reputational score or lack therefof.

What I don’t get is the protection of the endpoint hinges on a user response.  The demo I saw of Quorum presents a user with a warning screen.  The screen tells them how many people in the Norton community have used the file – few (less than 10), very few, or unknown – and presents the user with three choices:

  1. Decide later (the Scarlett O’Hara I will worry about that tomorrow option)
  2. Remove this file from my system
  3. Run the installation of the product anyway

So essentially the same user that got the endpoint machine into this mess is given a prevalence score and gets a choice of how to proceed.  In my opinion, prevalence protection is a smart idea right up to the reliance on the carbon based life form that clicked on something questionable or outright bad in the first place to now somehow have the wisdom and security awareness to properly respond.  

I am going to have to go with human nature here and guess that they will pick #3 – run the installation anyway.  Because human nature says: “If I clicked on it I want it and I don’t care about your fluffy risk rating”.  I actually think there is a direct correlation behind my claim – the more likely someone is to click on something dangerous, it will be proportionately likely that the same person would ignore any warning and proceed without care.   In other words, the more likely I need to be protected from my own actions, the more likely I will be to ignore the warning and continue on as if nothing had ever happened. 

 That is why I really believe that there has to be automated analysis and remediation behind this technology to really make it practical.  Just one man’s opinion.

My Briefing on Symantec Quorum Part 2 – Why I Think Triumfant Offers a Stronger Solution

Yesterday I detailed my impressions after being briefed by Symantec on their new Quorum product.  In summary, I was impressed with the implementation of the technology, but was not convinced that it solves the malware detection gap for enterprise customers, particularly those under the dynamic persistent threat scenario that see precise, well engineered and targeted threats on a continuous basis.

For such customers I believe that Triumfant’s approach to prevalence is far more applicable and practical.  When Triumfant scans an organization’s endpoint population, it builds a rule in the Adaptive Reference Model for every piece of software it discovers, along with information about the files and other elements associated with that software.  In other words, the model builds a functional whitelist that contains prevalence data specific to the organization and not based on the collective wisdom of a community.  And you can build models that address the entire endpoint population, or build models to specific groups of machines as appropriate.  The model is refreshed weekly to ensure that it accurately represents the desired evolution of the endpoint population.

You do not have to tell the model what is acceptable in your specific environment, it learns it.  You can, however, build policies and explicitly define authorized applications through a wizard driven interface.  If there is software already on the machines that ultimately is not in the desired list of authorized applications and programs for the organization, then it is a simple act to build what we call a filter to exclude specific software from the model and therefore the whitelist. 

Once the model is built, any application or program added to an endpoint machine or server that is not in the model as an authorized application is called to the attention of the administrative console.  Resolution Manager synthesizes a situational remediation to remove the application from the machine and ensure that every change to the machine made as part of the installation is reversed.  The remediation can be configured to execute automatically, or be set to require confirmation by an administrator prior to execution.  Either way, no human intervention is required to write the remediation, and every remediation if fully reversible.

Because Triumfant sees all of the changes to the affected machine that were part of the unauthorized software’s installation process, it has the information necessary to build a remediation that removes the malicious code and all collateral elements from the machine.  Why is this important? The installed application could be a trojan horse or be desgned to make configuration changes to weaken the defenses of the machine. So if the install included a secondary malicious payload, Resolution Manager will see it and kill it.  If the install opened a port or changes a security configuration setting, Resolution Manager will see it reverse the process.  

Symantec allows you to build custom alerts based on prevalence data returned from the Symantec reputation database, but from what I saw it does not included automated remediation.  The information I saw from Symantec indicated that it was the role of the client to block a file when an unacceptable reputation score was returned.  Given we can’t teach users to not open suspicious emails or click through social engineering; this would seem to be problematic. And because the application must install for it to be checked by the Symantec product, removal of the suspicious executable and all associated changes to the machine becomes critical. That is why Triumfant would seem to offer a superior solution.

Finally, I would add that the capabilities I describe for Triumfant exist today and are up and working at customer sites – this is not a future. 

I want to take the time again to acknowledge the Symantec team and their willingness to share the details of the product, as well as reiterate my belief that this technology will serve them well in the consumer market.  But for large organizations, I do believe that organization based prevalence is more practical than a community based prevalence.  I also think that Triumfant’s remediation capabilities address a significant shortcoming in the Symantec offering. 

But more importantly, just how much of the detection gap does Symantec expect this to solve? By my calculations, Symantec added approximately 1,700,000 in new signatures in the first half of 2009. More than they added in 2008 total.  That equates to just about 9,000 new signatures a day.  McAfee noted in an entry in their Avert Labs blog that they were writing over 6,000 new signatures a day and they don’t count what is caught by their generic filters and heuristics. Will this catch 10% of the attacks already evading the other protections? 50%?  Unless Symantec thinks this will be a near 100% solution, there would still seem to be a gap.

And that is where Triumfant really stands out: we believe Triumfant closes far more of the gap than any alternative detection and remediation tool.  We would never stand up and say that we close the gap completely, but we think we can make a case that we are pretty darn close.  Because we track all of the changes to each and every endpoint, you would have to be able to construct an attack that does its malicious activity without changing  the attacked machine for Triumfant to not see the attack.  So while prevalence may close part of the gap, why have a gap at all?  And for you folks that are not Symantec customers and like the idea of adding prevalence to your existing protections, we can do that for you and – a lot more – by providing the perfect complement to your antivirus software regardless of the vendor.

My Briefing on Symantec Quorum – Impressed But Not Convinced

On July 9 I wrote a post about Symantec’s soon-to-be-released reputation based technology they are currently calling Quorum.  My post was a bit tongue in cheek asking how something unknown could have a reputation, but it appeared to have been taken seriously by the folks at Symantec who pinged me back on Twitter and offered to help me better understand the product and the value of the reputation based approach.  I took them up on the offer and one of their product management folks walked me through the technology.

First let me say that I respect the earnestness and professionalism that the Symantec people had in seeking to correct what they thought were my perceptions of the product.  In return, I will refrain from providing any details of what they shared with me as they are rolling out the product as we speak and I certainly do not want to inadvertently include any information that they have not yet taken public.  I don’t want to damage my reputation score.

What I will share is my general conclusion: which is that while I was impressed with the technology I was not ultimately sold on was the benefit to larger enterprise customers that have to stand against a barrage of precision guided exploits and new attacks on a daily basis. 

The Symantec solution is extremely complete and obviously very thoughtfully constructed.  They have clearly considered a lot of the angles in reputation based technology, including safeguarding against methods to artificially influence reputational scores.  In my opinion, the technology will be a good addition for consumer customers and small businesses that should benefit from reputational comparatives given they have a small number of machines or only one machine to watch.   It also allows Symantec to leverage their large user base as well as integrate and showcase their data storage capabilities to their security customers.  Like I said – in regards to the implementation I was really quite impressed. 

For large enterprises, particularly those customers who are under what we call the dynamic persistent threat scenario, I am not convinced that the prevalence data from a broad community will fill the existing gaps in malware detection. These are organizations that fend off deliberate and precisely targeted attacks designed to extract critical financial data or confidential strategic information by exploiting new attack vectors, recently identified exploitable flaws, or variants of known attacks to evade the traditional defensive software that relies on prior knowledge of attacks for detection.  While these customers might find a community based prevalence score interesting, they frankly are of a profile where such a score – or lack thereof – is not sufficient to make determinations of the potential malicious nature of applications.  The fact that it has been installed in a number of other organizations does not mean that it is acceptable to be installed on their endpoints.

I am grateful to the Symantec team and their willingness to share the details of the product.  I exited the process very confident that while the reputation based technology may help Symantec in the consumer market it has not addressed the shortcomings their tools have in detecting attacks where there is no prior knowledge or the dynamic persistent attacks that many organizations battle on a given day.  In fairness, I am admittedly biased and these shortcomings are not specific to Symantec and are shared by endpoint security vendors as well as their customers. 

Tomorrow I will make my case and detail how I think the Triumfant approach is more applicable and ultimately, more practical.  For example, we already have organization specific prevalence baked into our model.  And we build automated remediations for what we find.  Have a look tomorrow and see if you agree.

McAfee Publishes Numbers On Aggressive Malware Growth

McAfee has just posted some number of their own regarding the growth in new attacks (and the subsequent need for new signatures) via a blog post by McAfee Avert Labs.  In that post, McAfee says that the number of new attacks is three times the rate over the same period last year, and that the number of attacks for the first half of the year nearly eclipsed the total for all of 2008. 

We have been leveraging the Symantec numbers for our Worldwide Malware Signature Counter, and it is nice to see that the McAfee numbers back up our basic thesis.  McAfee reports their numbers a bit differently from Symantec, in that McAfee excludes those attacks that were picked up by generic filters and heuristics (much more on that next week).  This makes the McAfee numbers smaller in total, but they represent the same aggressive growth curve as Symantec’ numbers.  For example, if you read between the lines, McAfee saw roughly 500,000 new threats in the first half of 2008, nearly 1,000,000 in the second half, and 1,200,000 in the first half of 2009. 

There has been some interesting new language from the AV vendors regarding the aggressive growth of new attacks and the growing strain to build signatures fast enough to protect their customers. Symantec is trotting out their Quorum whitelist/reputation based technology as the cure, but it remains to be seen if it can really close what these numbers illustrate is a large and growing detection gap.  In shifting the emphasis on the Quorum technology, Symantec is publicly falling on the signature sword.  In the Quorum press release, a Symantec executive is quoted as saying: “Looking at the sheer volume of infected systems in the world, one thing is resoundingly clear: basic security protection is not good enough.” 

Clearly the “elephant in the room” problem has gotten large enough that the AV vendors can no longer act like it is not there.  Because if I interpret the language in this blog post properly, the numbers presented by McAfee are those attacks that fell through all of their nets – signatures, generic filters, and heuristics – at a rate of 6,000 per day.  I do not single out McAfee as I am quite certain that these numbers are representative  just how much is getting through the existing endpoint security defenses of all of the AV vendors, Symantec included.

When you point out a problem – such as the unstustainable nature of the reliance on signatures – publicly the way that Triumfant has done, you draw criticism along the lines of fear mongering or that the sky is falling.  But the McAfee and Symantec research numbers present an objective case and the language of the AV vendors in the press clearly support our position.   Half the problem for us was creating awareness that there was a problem and that it was sizable and growing rapidly. 

We do agree with Symantec in that the IT security market is in need of new thinking and a new approach to counter this growing threat, and we think Triumfant is that new thinking and approach.  Now that the numbers support the story and even the AV vendors are recognizing the problem, we invite you to take the next step and hear what Triumfant has to offer (today, not a future release) as the solution to this problem.  I am willing to go on the line and say that you will at a minimum find it interesting and enlightening and won’t feel like it was wasted time.   We think we have filled the detection gap in a way that is both powerful and elegant, and is already addressing the problem for real customers today. 

What do you have to lose except the exposure to what McAfee says is 6,000 new threats per day?