Detection is the Horse, Investigation is the Cart – Use in That Order

I received some interesting responses from my last week’s post (Incident Detection, Then Incident Response) so let me try to answer them all collectively.

No, my post was not a knock against incident response (IR) or forensics tools.  I believe we are getting things out of order.  It is about detection first.  Better analysis? Good. Better Response. Good. But it all starts with breach detection.  In fact, if we had better breach detection, organizations would actually get more value out of their IR/forensics tools.

The inability of organizations to detect breaches is easily explained.  The picture below is my attempt to illustrates what I call The Breach Detection Gap.  This gap exists  between the numerous layers of prevention solutions and IR/forensics tools leaving organizations unable to detect breaches at the point of infiltration.

The IT security  market has been fixated – technically and emotionally – on prevention. Hence the numerous “usual suspects” on the left side of the breach.  I think my position is clear (cystal) that a prevention-centric strategy is doomed to failure.  Tradecraft relentlessly and rapidly evolves to evade any gains in prevention, and targeted attacks and the Advanced Persistent Threat are engineered to evade the specific defenses meant to defend their target.

IR and Forensic tools provide deep insight and valuable analysis to the breach investigation process, but are only brought to bear after the breach is detected.  Unfortunately, this is where most organizations spend the meager budget slice that is set aside for post infiltration.

The Breach Detection Gap is the critical exposure between prevention tools and IR/forensics tools that leave organizations without the means necessary to detect breaches in real-time.  Obviously, without detection there can be no timely response.  Which is my point of last week’s post: re-packaging IR tools as the solution for breach detection problems is not the answer.  The answer must start with faster and more accurate detection.

Someone also asked why I don’t name names.  I try to write this blog to stimulate thought and while I unashamedly say where Triumfant solves specific issues I try very hard to keep this from being an ongoing advertisement.  I also have never believed that there is any value from directly speaking in a negative manner about any other vendor.  There are some good IR/forensics tools in the market that are very hot right now, and when products get hot, the market begins to act strangely around them.  My post was not a knock on those products, but on the efforts I see in the market to position those tools with professional services as the solution to the Breach Detection Gap.  Make no mistake, the organizations around these hot products and event the vendors behind these products see this as a chance to sell professional services projects to hunt down breaches.  I will leave it to you to figure out who those vendors are, but I think in most cases the answer will be easily discerned if organizations resist the hype.

What I did not say in last week’s post is that Triumfant is positioned to detect breaches in real time.  There are ample posts that address that directly as well as a new whitepaper on our site, so I won’t go into details here.   I will say that while heuristics, behavioral, and IPS/HIPS are also being directed to the problem, I think that Triumfant’s use of change detection and the analysis of change in the context of the host machine population is uniquely suited for the role of breach detection.  You get rapid detection (real-time), and within minutes we provide detailed information to help formulate an informed response, and we custom-build a remediation to stop the attack and repair the machine.  That is rapid detection and response.

And while Triumfant provides a wealth of IR/forensics data, we fully endorse the use of IR/forensics tools to provide the full range of post-breach investigative work.

But it all starts with detection.

In 10 Days, the Mac Safe Haven Becomes a Botnet Spewing, APT Vulnerable OS

In rapid succession, the IT security world, not to mention the perceived cocoon of safety for Mac users, was rocked by two announcements.  On April 4, Russian antivirus company Dr. Web announced that they had discovered a Mac Botnet, called Flashback, and that the bot had infected 600,000 machines.  About ten days later, Kaspersky announced the discovery of a backdoor trojan called Backdoor.OSX.SabPub.  This attack leverages an exploit that uses malformed Word documents to deliver malware that opens a backdoor that can be used for advanced, persistent attacks.  Holy APT Batman!  Perceived safety to botnet to advanced persistent threat in 10 days!

Oh the shame.  The Mac went from safe haven to botnet spewing, APT exploitable platform tied to three-year old vulnerabilities before our very eyes.  As I tweeted, the heads of the Mac fanboys and the APT crew were simultaneously exploding.  Mac users were sent to various sites to download software to check their machines for Flashback like common Windows XP users.  I could not help but wonder if some enterprising bad guys had set up malware delivery disguised as Flashback checkers – wouldn’t that have been ironic.

I am really just having some fun here.  I take no joy in the Mac becoming a target, although it is good for business.  I am also not on some war against “smug” Mac owners because I have made the jump myself.

For me, the folklore/mythology of the Mac world as a safe haven from malicious attack reminds me of a scene from the classic movie and personal favorite, Butch Cassidy and the Sundance Kid.  In this scene, Butch and Sundance have fled to Bolivia and have taken a legitimate job guarding the payroll for a mining company.  At the beginning of the scene they are riding with the old, hardened mine boss (played perfectly by the great character actor Strother Martin) and begin to argue where the inevitable ambush will occur.  The mine boss responds disdainfully: “Morons. I’ve got morons on my team. Nobody is going to rob us going down the mountain. We have got no money going down the mountain. When we have got the money, on the way back, then you can sweat.”

Mac users, I hope you have enjoyed the ride down the mountain.  The recent Mac malware news just means that the downward portion is over, and now that there is a critical mass of Macs plugged into the networks and systems where the money lies. It is time for Mac users to sweat.

We could engage in what I am sure will be an animated conversation about the superiority of the Mac OS and the inherent vulnerabilities of Windows, but I contend this was all about opportunity.  Sure Windows machines were likely the road of least resistance, but malware writers have proven to be a resilient and industrious bunch and repeatedly rise to find a way around every barrier put in their path.  So now that the opportunity has arrived – what the adversary wants is on or accessible via the Mac – the Mac OS barriers will also be breached.

I should point out that Mac users are not finished with their journey into the seedy underbelly of IT security.  Not surprisingly, the sales of Mac AV software has gone way up.  Wait until the Mac people connect the dots that the same crew that discovered the malware also sells them AV software.  Of course, that AV software will at least partially return their cocoon of safety, until they find out that motivated adversaries will drive around their new shiny AV software like a traffic cone on the interstate.

I hope they enjoyed the ride down the mountain.

Digitally Signed Malware Proves Again That Attacks Get Through Your Shields

So what, Triumfant guy, exactly gets through my shields?  You tell me I will be breached and you give me statistics, but I have AV, whitelisting, deep packet inspection, and every other acronym and buzzword in place. Oh yea, and I have “the cloud” (pause for tympani emphasis) providing me prevalence information and other cloud-based stuff.

Well, digitally signed malware gets past your protections.  Not according to me, but according to several sources – Symantec, Kaspersky, AlienVault and BitDefender – cited in a March 15, 2012 PC World article “Digitally Signed Malware Is Increasingly Prevalent, Researchers Say”.

It is the blackhat version of “these are not the droids The Droids You Are Looking Foryou are looking for”, using the certificates to get the malicious code waved through.  Some of the first evidence of this technique was found in 2010 in the analysis of Stuxnet.  The PC World article provides evidence that the technique is showing up with increasing frequency.  The article tells in good detail how it works and what protections it can evade, including whitelisting.

This technique is illustrative of the ongoing battle between good and evil in IT security.  Operating system advances in Windows 7 and other OS versions were thought to advance the security of systems, and the adversary then takes the very techniques used to make the systems more secure and subverts them to find new ways to deliver malicious code and evade protections.  I have no interest in impugning the efficacy of prevention software and I have never said to turn off protection software.  What I have said consistently is that attacks will get through your shields.  Here is yet another example of how, and demonstrates that the adversary will always find a way to get through.  No FUD here – I would point out that every vendor cited in this story is a protection software vendor.

This story also illustrates that there are no silver bullets in protection.  Prospects often cite the use of whitelisting tools as their raison d’etre  of why they do not need something like Triumfant, but here is a clear example of how such tools are being evaded.  If you need more, there is a video from Shmoocon that shows multiple techniques for evading several whitelisting tools.  Yet another silver bullet falls short. I am not singling out whitelisting – it is just the current “It” tool of IT security.

Lastly, it is illustrative of how the foundations of trust have become less…well…trustworthy.  I have seen the validation process of a certificate authority up close, and let’s just say I am not shocked to know that malware writers can obtain certificates with false identities. With the RSA breach and other certificate authorities being hacked, the foundation of trust was already showing cracks.  Now we see examples of how trust can be subverted using this technique.

So if this technique essentially waves malware through your shields, how are you going to detect the infiltration?  That is where Triumfant fills the gap, detecting the zero day attacks and targeted attacks, including the advanced persistent threat, that infiltrate your endpoint machines and servers.

I once had a product manager from another company disdainfully tell me “when you find something that gets past my shields, you call me”.  I am looking for his number as soon as I finish this post.

Targeted Attacks Make Remote Adversaries Malicious Insiders

“Wow, your tool would be great against malicious insiders!”

This is a common conclusion made by those introduced to the Triumfant solution.  That is because instead of looking for applications or malicious executables, we detect malicious activity through change, whether a threat actor working programmatically creates the change or a malicious insider directly makes the change.

The term “malicious insider” has been gnawing at me since I delivered a short presentation for the Intelligence and National Security Alliance Innovators Showcase last week.  My new slides had several screen shots from the Poison Ivy Remote Administration Tool (RAT) that we use in demos of the Triumfant product.  It was interesting to see the reaction to those screen shots as people grasped in a very graphical way what it meant to “own” a machine.  I realized that perhaps while people have intellectually grasped what a RAT can do, they might not have fully appreciated the term “own” until they actually saw one in action. (More on RAT tools in the previous post)

Today’s attacks are not smash and grab operations – they methodically evade network and endpoint protections to establish a long-term and comprehensive presence on the machine.  These are carefully crafted incursions onto target networks that rely on persistence and stealth.

In short, they turn the outsider into an insider.  This of course is not news to those in infosec, but to the people we serve, this is an idea they are still wrapping their head around these sophisticated targeted attacks.

Once a RAT is in place, the hacker has the same access as if they were looking over the shoulder of the machine’s user.  The user literally guides them through the applications and systems on the network, providing them user IDs and passwords along the way.  This allows the hacker to spread their influence to other places in the network until they are able to access their targets.   Time is on their side, as every statistic says that they will have at least a month and on average six months to identify and exfiltrate the intellectual property or sensitive data they seek.

Attacks rarely start at the machine that holds the targeted information.  Hackers now patiently gain access to the network where they can, and then stealthily move about until they find what they need.  And new Advanced Persistent Threats like Duqu illustrate that hackers are now using sophisticated attacks to gather all manner of information to then plan their payoff attack.  As I said in the previous post, these attacks put the adversary in your boardroom, laboratories, production lines, and CFO’s office.

If six months and virtually unlimited access does not qualify the hacker as an insider, I do not know what does. Recruiting physical insiders is a long and costly process and smacks of too much Mission Impossible.  And even well placed insiders may have trouble moving outside of their areas of responsibility.  Why go through all of that risk and effort when an outsider can easily become an insider.  If the operation is discovered, the outsider simply moves to the next target.

There is another aspect to being an insider: once you are inside, all of the security measures designed to keep you an outsider are now irrelevant.  All of the carefully crafted shields an organization has in place are all pointing outward and are not equipped or designed to catch the work of an insider.  Once these shields are evaded they are no threat to the insider.  Statistics from the 2011 Verizon Business Data Breach Investigations Report say that less than 6% of data breaches are discovered by the organization’s IT shop.  That sound’s like a pretty wide gap that requires some new thinking to me.

The answer to the original question is yes, Triumfant rocks against malicious insiders.  All types.

RSA Conference 2012 Fearless Forecast – The Cloud of FUD

Next week, something insidious and life-choking will settle over the San Francisco Bay area and threaten everyone with confusion, nausea, and full loss of body hair.

The cloud of FUD.

For you South Park fans, yes, this is far more dangerous than the Cloud of Smug introduced in one of the classic South Park episodes (The Perfect Storm of Self Satisfaction). In the episode, the South Park residents begin to purchase hybrid cars (the Toyonda Pious) in large numbers, and their self-satisfaction in their eco-friendly ways creates a dangerous cloud of smug.  Unfortunately, the South Park cloud collides with two other clouds of smug, one from the general self-satisfaction of the SF Bay inhabitants and a rogue cloud from George Clooney’s Academy Award speech.  This creates the perfect storm of self-satisfaction with catastrophic results, destroying San Francisco and causing general havoc in South Park.

The RSA Conference is next week, and the amount of FUD in any normal RSA week can be problematic.  But this year, the IT security world is at an interesting crossroads.  The underpinnings of trust have been called into question through breaches of companies like Diginotar, and more recently, VeriSign.  Analysis released last week called into question encryption algorithms used by RSA, who is still reeling from a highly public breach last year. Studies indicate that breaches are on the rise, and targeted attacks (including the Advanced Persistent Threat) are hitting their mark with increasing frequency.  And we have no idea how many breaches are yet undiscovered and when we do discover them, we lack the tools to fully assess the damage.  The public disclosure of the VeriSign breach included language from VeriSign management that they were still not quite sure what had been stolen, in spite of the breach occurring in 2010.   Attacks like Duqu were illustrative of the growing sophistication in data gathering techniques to build even more sophisticated follow-on attacks.

We have entered a new phase in IT security to be sure, and all of this uncertainty will amplify the FUD volume to deafening levels.  That is because while there are several innovative companies offering real solutions to these new problems, the majority are scrambling.  When companies scramble in the IT security market, the result is a Perfect Storm of Self Preservation.  Those who lack real answers will look to duck and cover, and the predictable result will be epic volumes of FUD with a healthy undercurrent of smug.

Seriously, we should consider renaming the RSA 2012 exhibit area FUDapalooza! I am not talking about the usual “hamster wheels of pain”, “yes, I do that” (before a question is asked) level of FUD.  This will be highly advanced, super concentrated FUD.

For example, everyone, including the nice people that serve old, stale sandwiches in the lobby for $18, will have “The Solution for the Advanced Persistent Threat”.  Everyone will have the “Next Generation of Threat Protection” and “Your Weapon for Cyber Warfare”.  Companies that went the M&A route will have the “First Truly Comprehensive Security Suite/Platform”.  The large, “usual suspect” companies with the huge booths at the center of the floor will promise to plug the massive gaps that studies now show their own products to have.

I remember my first RSA Conference in 2005.  I was immediately struck by the signal to noise ratio (very little signal, copious amounts of noise) and lack of clear messaging and differentiation on the exhibit floor.  One of the more popular posts for this blog was about the animals you will see at RSA.  I can only imagine what 2012 will be like.

At the end of the South Park episode, Kyle points out to the citizens that driving a hybrid is really a good thing, but they have to learn to drive them without being smug.  The townspeople go back to their old gas guzzling cars, saying that “it’s simply asking too much”.  The RSA Conference could be an excellent place to explore ways to meet the new challenges we collectively face today.  Unfortunately, I think for most of my vendor comrades “it’s simply asking too much”, and most will instead take the Gladiator approach and unleash FUD hell.

The Cloud of FUD is coming.  Bring your Hazmat suit.

Targeted Attacks Versus Advanced Persistent Threat – Pragmatic Versus Dogmatic

In some circles of IT security, debating the exact definition of what constitutes an Advanced Persistent Threat (APT) is far more incendiary than debating politics or religion.   I was forced to wade into these tumultuous waters this week as I was making updates to the Triumfant Web site.   Specifically, I was curious to see if there was some industry consensus as to the dividing line between the two classifications. Silly me.  I should have known better.

The volatile nature of the definition of APT makes the dividing line between targeted attacks and APT equally volatile.  The industry has not settled on any one dimension to distinguish and APT attack, much less a specific point on that dimension.  For some, APT is determined by the nature of the attack, or the target of the attack.  Some, most notably Richard Bejtlich (@taosecurity) define APT by the threat actor.

After some research, it became obvious that the one thing the debate needed was yet another attempt to differentiate APT attacks and targeted attacks, and being shallow and self-centered, I knew I was just the guy for the job.  My simple classification came down to pragmatic (targeted attacks) versus dogmatic (APT) and actually incorporates most of the elements of the debate.

At the high level, I consider APT attacks as a subset of the broader category of targeted attacks as both are attacks written to perform a specific purpose against a specific target.  Both value stealth and seek long-term infiltrations.  Both involve sophisticated adversaries that often use many of the same techniques.  Given the two categories are not exclusive, what I am attempting to capture is the point where a targeted attack becomes an APT.

Targeted attacks are pragmatic because their motivation, and therefore their approach and behavior, lies in monetary gain.  A targeted attack is likely designed to extract confidential information or intellectual property.  It is conceivable that the attack could be disruptive, but pragmatically, disruption does not provide a return on investment.   Targeted attacks value stealth and long-term infiltration, but only to the point where they serve the pragmatic need.   Not quite smash and grab, but not the longer-term persistence sought with APT.  Targeted attacks rely heavily on techniques that leverage human nature (social engineering) because the adversary lacks access to the human-gathered intelligence available to the APT threat actor. Finally, a targeted attack may be reusable against other targets, albeit with some modification and mutation of the malware.

I use the term dogmatic to describe APT attacks because APT attacks are largely driven by emotional/philosophical motivations, primarily politics.  This places higher value on stealth and persistence than a targeted attack because it enables the adversary the freedom to alter post-infiltration activity to respond to evolving external events.   This is the proverbial low and slow approach that places high value on maintaining an established presence in the targeted system or network.  APT attacks may also be broader in their impact to the targeted organization because disruption may provide the same political impact as exfiltration.  APT attacks often consist of multiple parallel attacks to ensure infiltration and ensure that discovery of one path does not cut off presence in the network.   That is because a pragmatic adversary may be able to move onto the next target, but the target for a dogmatic adversary is dictated by the politics of the moment.

I am going to be very candid and say that I really have no real emotional or professional stake in this debate.  Triumfant excels at detecting these attacks, and the dividing line has no affect on that capability.  I simply was creating a web page on targeted attack detection and a separate page for APT detection, and I was doing the due diligence to be as accurate as possible.  Why separate pages? Both terms (“targeted attacks” and “advanced persistent threat”) are frequently used search terms, so it was all about providing information to those who get to the Triumfant site through organic search.

So there is my take on the debate.  Not sure if the pragmatic versus dogmatic designation helps, but it resonated with me, so who am I to not feed the fire?


VeriSign Breached – Who Can You Trust Redux

It was reported by Reuters today (“Key Internet operator VeriSign hit by hackers“) that VeriSign has disclosed that the company was hacked in 2010.  This is significant at many levels.

First, VeriSign essentially handles the credentials for over half of all Web sites, specifically sites ending in .com, .net and .gov.  VeriSign executives could only say that they “do not believe” that the critical domain name services, leading many to speculate that VeriSign does not yet know the extend of the breach.  And even if the domain name services were not compromised, compromise of any of VeriSign’s other services could still represent significant risk to a very large number of companies and government agencies.

Given that VeriSign has not been forthcoming with details and frankly does not seem to know yet the full extent of the breach, the security of an enormous amount of Web sites is in question this morning.  I am not sure that this can be understated.  Depending on what we learn about this breach, the tectonic plates of online security may have just shifted significantly.

Second, the VeriSign breach is a huge blow to the topic of trust on the Internet (see  the blog post “Certificate Authorities Hacked – So Who Can You Trust?“).  This trust was already significantly impacted by the RSA breach last year and the compromise of several certificate authorities (CAs) such as DigiNotar.  But the aggregate affect of these breaches, in my opinion, is dwarfed by a compromise of VeriSign.  Consider that the “s” in “https” is based on Secure Sockets Layer (SSL) certificates, the majority of which are issued by VeriSign.  Suddenly the ubiquitous lock icon and green indicator of  web site trust suddenly do not feel so secure and trustworthy.  The past months have been filled with questions about the trustworthiness of SSL, and this breach will pour gasoline on that fire. In a broader sense, the article points out the RSA and VeriSign attacks are designed to undermine the fundamental underpinnings of authentication.  This puts all transactions – business, government, personal – at risk.

Third, the VeriSign breach came to light in a 10Q filing with the SEC that listed the breach in accordance to the new SEC guidance on breach disclosure.  Reuters did a search of such disclosures and found the VeriSign admission.  Without the SEC guidance, this breach may never have come to light and the companies that trust the integrity of VeriSign’s services would have never known.  I draw the conclusion that there was no communication of VeriSign to their customer given that the CTO of VeriSign at the time of the breach learned about the problem from Reuters.

The potential impact of this breach could make this event the tipping point in the call for more strict guidance and perhaps even legislative action in regard to breach disclosure (see “Prediction Regarding Data Breach Detection – Soon to be a Regulatory Requirement“). Proponents will have a field day with the idea that VeriSign may have never disclosed the breach without the SEC guidance.  But opposition to such action will also use the event as an argument against such action. The article intimates that the breach is a persistent attack done by a nation-state, or an Advanced Persistent Threat attack.  Such an attack at a company such as VeriSign has far reaching impact on national security, so there are those who would not want the attack disclosed before there was reasonable time to perform analysis, attribution, and potentially launch a counter attack.  Mix this attack in with a presidential election year and I predict the skies will darken will calls and counter arguments for legislation.

Fourth, this event may finally take many over the emotional hump  of clinging to the hopes that 100% prevention is still possible (see “The Emotional Barriers to Embracing the Presumption of Breach Doctrine“).  The article quotes security consultant Dmitri Alperovich as saying “prevention is futile”.  Those who have clung doggedly to prevention in the face of mounting evidence will find it hard to continue to do so.  It is okay.  Those of us who have already accepted the inevitable are here, waiting for you without judgement.  Just let go.

Fifth.  I will have much more to say about this subject, but notice that although the breach happened in 2010, VeriSign still does not know the extent of the damage.  There were even intimations that they may not have completely eradicated the adversary from their systems.  This is proof to my ongoing statement that organizations are not equipped to detect, analyze, and respond to breaches.  Trust me when I say I have much more to say on this topic in the very near term.

Watching this story unfold should prove to be quite interesting.  Quite interesting.