VMware vCloud Director Support

To quote BusinessWire, we’ve just announced another world first. At VM World today we announced our support for VMware vCloud Director in LogLogic 5. Want to see it in action? Press play below…

Posted by Andy Morris on August 31, 2010 in Cloud Computing , Innovation , LogLogic News , SaaS , Security | Permalink | Comments (0)

Dark Clouds.

By Dimitri McKay

Several years ago a new buzzword was formed. “The Cloud”. This was a familiar concept to anyone using “web mail” whether they knew it or not. Email was being offered as a service on the internet. So if you’ve used hotmail, Gmail or Yahoo mail, you’ve used cloud technology. But it goes further than that.

Software as a Service (or SaaS) are cloud based applications not in your local datacenter, but rather, off on servers owned and maintained by some other organization who have a very tightly controlled datacenter, and offers these services on the internet.

The processing, the storage, and the data are all part of a large grids and data centers where distributed computing takes place. We’re talking about hundreds of thousands of server farms which can much more effectively manage the processing power of a service, application or company far better than a single server or node of servers. In fact, economically, cloud computing is far cheaper than traditional computing paradigms.

But there’s trouble in paradise.

In a recent survey 60% of respondents expressed some reservations about moving to cloud computing technology, which, considering all the talk of how fantastic Cloud Computing is, it didn’t appear to be on the roadmap of most companies. In fact, 34% of respondents said that cloud was not even a strategic move for their companies. Why? Because according to 26% of those surveyed, said that they had some form of fear over the risks, security, control and transparency in particular. Many felt that the lack of controls created massive security risks.

The biggest question has been, thus far, how do we offer data security and transparency in the cloud, when frankly, it’s the job of the cloud providers to offer that visibility related to those practices.

Even the early technology adopters surveyed said that they wouldn’t be adopting the cloud just yet. So here are the pro’s and con’s of Cloud Computing.

First up, the benefits. Cost. To start, your local IT people who take care of local application servers, local infrastructure to maintain access to those application servers, and security folks to tie it all down can now focus elsewhere. Your critical applications are moved out to the internet along with the cost of maintaining the infrastructure and server hardware/software/operating systems. The Cloud vendor will have their own gear, infrastructure, and employees. So you can immediately readjust budgets and priorities.

In addition, cloud computing is super cheap. If you’re growing an application and have an active growth projection model, adding bandwidth, adding processing, adding storage can be dirt cheap. Cloud vendors offer computing power at cents on the dollar.

Lastly, the cloud is hugely scalable. No longer do you have to wonder if you have the horsepower, the bandwidth or the staff to support your growth efforts. Instead, you can build out capacity as you need it. At very low cost.

Next we’ll go in to the pro’s and con’s in more detail.

Posted by Andy Morris on August 11, 2010 in Cloud Computing , Dimitri , LogEd , Risk Management , SaaS , Security | Permalink | Comments (0)

Surgeon Sentenced to Federal Prison for Violation of HIPAA

By Barbara Rogan, LogLogic General Counsel

Quick, what are the first five things you think of when I say “cardiothoracic surgeon.” I bet convicted criminal wouldn’t even be in the top 100…

On Tuesday, Huping Zhou, a cardiothoracic surgeon formerly with UCLA Healthcare system, was convicted of 4 misdemeanor counts of violating HIPAA (here’s link to USAO press release: http://www.justice.gov/usao/cac/pressroom/pr2010/079.html). Apparently, after Dr. Zhou was told that his services were no longer needed for unrelated reasons, Dr. Zhou couldn’t resist peeking into the medical records of his supervisor, many of his colleagues, and several celebrities, including Tom Hanks, Drew Barrymore, and Arnold Schwarzenegger (http://www.cbsnews.com/8301-504083_162-20003669-504083.html) without legal or medical reason. For his misdeeds, Dr. Zhou will serve four months in Federal Prison for his crimes.

Just for fun, let’s change the facts a bit… What if Dr. Zhou, instead of working for UCLA, worked for a third party service provider who had a contract with UCLA? Prior to 2009, Dr. Zhou would likely have not faced prosecution as he would have been the employee of a Business Associate (i.e. an entity that provided services to health care providers) of a Covered Entity. Lots of lawyerly gobbledy-gook in that, but essentially it means, that Dr. Zhou would not be heading for prison.

Seems a bit silly to change the outcome based on who Dr. Zhou worked for. Congress agreed.

In 2009 the HITECH Act extended the criminal liability portions of the HIPAA act to such business associates. Now, even under the scenario where Dr. Zhou worked for a Business Associate, he would be subject to criminal liability the same as if he worked for a Covered Entity.

We here at LogLogic have put together a summary of the key provisions of the HITECH Act (http://www.loglogic.com/solutions/compliance/hitech.php) as well as a white paper (http://www.loglogic.com/resources/white-papers/hitech-act/) on how our Log Management solutions can help. The information we have gathered and summarized is of value to any company who is a Business Associate or may be asked to be a Business Associate in the future.

Posted by Bill Roth on April 29, 2010 in Healthcare , Legal Nerd , Security | Permalink | Comments (0)

Do your "Traffic Allowed" logs sing?

By Christophe Briguet, Log Maestro

In his blog post series "The Best Defense is a Good Logfense" Gorka Sadowski described how to fortify a ‘defense in depth’ strategy with logs. The release of our Log Source Package #16 (LSP) gives me the perfect opportunity to start a new blog series offering some concrete examples of Gorka's claims.

The newest LSP adds support for numerous new log sources including Fortinet, Microsoft SharePoint, McAfee ePolicy Orchestrator (with VirusScan), IBM DB2, Oracle DB server and Microsoft SQL Server. As you may know, these logs are notoriously hard to understand so we’ll start this series with something a little more comprehensible, the “Traffic Allowed” (TAL) log.

The TAL is the most common, the largest and the most overlooked of all the firewall logs. It records all activity that happens on your network legitimate or not. This log is also known as "build ... connection", "packet accepted" or "access-list ... permitted".

Firewall devices such as the newly supported Fortinet ones, produce logs for all accepted network connections that can tell you many things about your in- and out-bound activities. We’re going to use the TAL to perform the below six basic operations:

1. Confirm malicious activity
2. Detect policy violations, misuse, or the bypassing of internal controls
3. Detect DOS or an unusual number of connections
4. Detect suspicious traffic destinations
5. Create activity trend reports for the Operations team
6. Optimize rules to improve performance

Confirm malicious activity

Either after the fact during forensic analysis of your network, or during the real-time correlation of events, the TAL can be used to confirm specific connections. When correlated with an IDS it allows you to track activity of a specific IP address. Using a closed loop process you can re-create events and determine if an original alert was genuine or a false positive.

Detect policy violations, misuse, or the bypassing of internal controls

The TAL can be used to check for any unwanted protocols or services that the firewall should have blocked but didn't (e.g. IM services, P2P, etc.).

To complement the firewall filtering policy, traffic allowed logs can be used to detect misuse of production servers. For example, they can show allowed traffic from a critical Windows controller server that is not using a valid Windows-related protocol (e.g. AD, DNS, Kerberos, LDAP, etc.). You can use the same approach to detect outbound traffic on email servers that not related to SMTP, POP3 or IMAP. Or, knowing that SNMP can only come from a restricted range of machines, you can use the TAL to find rogue mail servers. I could go on about proxy servers, app servers etc. but you get the idea.

Most organizations have clearly defined Acceptable Use Policies (AUP) that outline how an individual can interact with the Internet, e.g. with these policies being automatically enforced by Content Gateways. It is possible to detect the bypassing of these gateways and the associated AUP violation, by searching for HTTP and HTTPs traffic not emanating from gateway. In particular, the HTTP POST request is often used by malicious software to connect to various PHP files on a control server.

Detect DOS or an unusual number of connections

Denial Of Service attacks or a worm out-break can be detected by analyzing the TAL. For example, an increasing and unexpected number of ‘accepted connections’ may be the consequence of malicious activity or security incident. In this case, with the traffic not being blocked by the firewall, the attack or propagation should be considered a priority.

Detect suspicious traffic destinations

The TAL can used to detect traffic with a suspicious location. For example, it is often illegal for some organization to communicate with countries that are under US policies and embargoes (e.g. Afghanistan, China, Congo, Cuba, Cyprus, Eritrea, Haiti, Iran, Iraq, North Korea, Lebanon, Liberia, Libyan Arab Jamahiriya, Sierra Leone, Somalia, Sri Lanka, Sudan, Syrian Arab Republic, Venezuela, Vietnam, Yemen, Zimbabwe). In some context, any traffic with one of these countries may be considered illegal.

Create activity trend reports for the Operations team

You can also use the TAL to create bandwidth trend reports, helping you better understand your actual network activity. (Who is talking with whom? With which protocol? How many bytes sent and received, etc.) Reports are usually of a ‘Top n’ list format but can be visually mapped to countries using a Geo IP database to help quickly spot unexpected traffic.

Optimize rules to improve performance

The optimization of firewall policies is critically important to provide high performance packet filtering particularly for high speed network security. ‘Traffic Allowed’ logs contain the ID of the filtering rules that match the traffic, which in turn can be used to rank rules and optimize performance.

Making a singing ‘Traffic Allowed’ log.

All the use cases described here will enhance your security operation and can be automated with a SIEM solution that provides correlation with an assets. But to be able to implement these smart correlation scenarios, you will need to build a strong and reliable log collection infrastructure that will provide the require scalability and storing capabilities to manage the voluminous (Tera-bytes, Petra-bytes?) Traffic Allowed logs. This log management layer will help to filter, correlate and selectively forward logs to the correlation engine according to use cases you want to implement. The best advice to make your logs sing is to start with Log Management and then grow up to SIEM.

Posted by Andy Morris on April 16, 2010 in Log Management & Intelligence , LogEd , Security | Permalink | Comments (0)

Logging on to a More Secure Workplace

People tend to see logs as overly complex myriads of data akin to that computer screen from “The Matrix.” Like that computer screen, though, logs play a crucial role in maintaining network security and stability, as well as identifying and isolating potential threats. Logs allow organizations to track massive amounts of data in a highly efficient and transparent manner. Thanks to logging, auditors can focus their attention on more pressing issues and have faith that log management software will alert them of any red flags.

This is especially important in an age where many experts say that the greatest security threat posed to large companies comes from within. For one our clients, a major telecommunications equipment maker, logs are used to track individuals from the moment they swipe into a building, walk through different security zones, enter a data center, log on to a terminal, take out data with a USB device, and finally swipe out.

Logs have also emerged as an important tool for analysts conducting forensic analysis of user behavior on given networks, allowing them to identify and neutralize potential threats. One client of ours, a university, was able to quickly recover sensitive data involving a number of students suspected of being involved in extremist activity. Based on the logs, authorities were able to detect information and websites the students had accessed, and with whom they communicated.

In this video, we showcase, albeit in a more abstract story, how federal agencies have used logs for forensic analysis:

Posted by Andy Morris on March 31, 2010 in Log Management & Intelligence , Security , Videos | Permalink | Comments (0)

Logs: Saving Lives and Fighting Crime, One Bit at a Time

It’s no secret that many of our clients select our log management appliances for compliance reasons – be it PCI, HIPAA, Sarbanes-Oxley, NERC or any number of other regulations and standards. But what they’ve universally told us is that they had no idea that logs could enable so much more! Forensic network analysis, troubleshooting network issues, even crime-fighting.

That’s why we’ve decided to highlight some of the more interesting customer stories we’ve collected over the years to give logs their proper due—we love logs and we want to show you why. Logs record and store massive amounts of data, allowing organizations to vastly improve the efficiency of their IT operations and security infrastructure. We hear as much from our clients, who include multinational corporations, government agencies, and nearly everyone in between. While logging may not be the sexiest IT topic out there, it could save your life someday. Don’t believe us? Read on.

One of the most impressive stories we have heard involved a C-level executive at one of the nation’s leading medical technology companies who was receiving threatening emails from an anonymous person. The company’s IT team was unable to identify the source of the emails even after an exhaustive search. Finally, they turned to the logs, which were able to pinpoint the source of the messages in just minutes. We thought this a worthy story to re-enact and kick off our “I Love Logs” series.

Another story that we found particularly interesting involved a government agency using log management data to identify doctors that prescribed Class 3 narcotics at abnormally high rates. Rather than wasting precious manpower (and even more of your tax dollars!) combing through massive amounts of data, the agency simply received an automatic alert anytime the system detected a red flag, and proceeded to take appropriate action.

On two separate occasions, hospitals that employ LogLogic technology used the logs to track down unauthorized users who accessed the medical files of famous celebrities. The unauthorized users were relieved of their duties, and the tabloids were without front-page stories the next day. All thanks to the logs.

Posted by Andy Morris on March 30, 2010 in Log Management & Intelligence , Security , Videos | Permalink | Comments (0)

Breach: Kneber Bot Attack

By Gorka Sadowski, Log Evangelist

Another hack attack hits the headlines http://tinyurl.com/yebvj8p

Big deal. This stuff happens every day now right?  Wrong. Not on this scale it doesn’t. The Kneber Bot has penetrated 75,000 systems, 2,500 companies across in 196 countries.  This is not a straightforward Trojan - a simple smash and grab. This one’s a game changer. 

Systems compromised by this botnet provide the attackers with not only user credentials and confidential information, but remote access inside the compromised network.  Just some of the data stolen includes:

• - 68,000 corporate log-in credentials
• - Access to e-mail systems, online banking sites, Facebook, Yahoo, Hotmail and other social networking credentials
• - 2,000 SSL certificate files
• - Dossier-level data sets on individuals, including complete dumps of entire identities from victims' machines.

Penetration of this scale and amongst such an esteemed group of public and private organizations - Merck & Co, Cardinal Health, 10 US Government Agencies - makes it is clear that no-one is untouchable to an ambitious,  determined and organized group of hackers. But what’s most startling is the lack of visibility about this particular bot.

Firstly we don’t yet know where it came from. Fingers have been pointed at China but there appears to be very little hard evidence. Next, we don’t actually know the extent of the damage. This apparently, is still being assessed, and affected companies notified. Moreover it isn't clear to what extent the attack has been contained.

What we do know is that it started in late 2008 in Germany. But that in itself begs another unanswered question. How can an attack using a spyware freely available in the Internet penetrate 75 000 systems Worldwide – and still go unnoticed for more than a year?

What is becoming ever more clear is that conventional malware and signature based detection systems are fast becoming inadequate for addressing the increasing sophistication of cyber attacks like the Kneber Bot.

So how can companies improve their visibility and protect themselves against these increasingly sophisticated attacks going forward? Well, regardless of the sophistication of the attack all computers natively generate electronic fingerprints. For every event that takes place in a computer or a network or a security system, or applications, databases or OS etc. a small record of that event is kept, it’s called a log. 

This is your electronic fingerprint. Just like a fingerprint, properly managed logs enable us to carry out forensics, and get us the visibility required to know exactly what happened, who did what, how the attack originated, how it spread, where are the attackers, what has been compromised.

So could the key to solving and preventing IT crime lie in properly managed logs? Could it be that log management could be of some use?

Yes, certainly. But the trouble is that with the explosion of corporate systems the number of logs has exploded to a difficult-to-manage number and few companies are truly geared up to manage them all – meaning that things inevitably slip through the net. Only companies using the most sophisticated log management systems such as LogLogic’s Open Log Management Platform which - with our new Quad-core hardware can monitor up to 250,000 records per second – can really hope to identify and act upon these new subtle, sophisticated and well-disguised attacks on their infrastructure.

The hackers’ game has moved on. We all need to be prepared to respond to this.

Posted by Bill Roth on March 11, 2010 in Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

Security Breaches: The Victim Will Get Blamed, and Worse

Blame the victim.  This was a common defense in sexual assault cases I helped prosecute when I worked as prosecutor.  Unfortunately this mentality applies not just to rape cases, but also to companies where critical data has been breached – even when the criminals are the ones stealing the data.

One of the biggest data breaches in recorded history hit Heartland Payment.  This is a bona fide case of the bad guys attacking networks and compromising critical data.  In Heartland Payment’s case, the data breach wasn’t found for many months and Heartland Payments has no idea of how many credit card numbers were jeopardized.  Potentially millions of credit card numbers, but no one knows for sure (or at least they are not saying so publicly).  To deal with the publicity and legal fall out, Heartland established a website (www.2008breach.com) to deal with the breach.  The bad guys were caught pretty quickly after the breach was discovered (see: http://www.informationweek.com/news/security/attacks/showArticle.jhtml?articleID=214303553)  and they have already pleaded guilty (see: http://news.cnet.com/8301-27080_3-10423008-245.html).  

But the fact that the bad guys were brought to justice did not exonerate Heartland .  Just this last month, Heartland Payments paid a settlement to American Express of $3.5 million for damages associated with the breach.  Amex apparently was the smaller of the three settlements Heartland will have to pay as they still have not settled with Visa or MasterCard yet.

Okay, so Heartland is a big company, but smaller businesses have been hit with law suits for failing to protect data.  RockYou, a Facebook app, was recently sued in San Francisco in a class action lawsuit (see: http://news.cnet.com/8301-1009_3-10423042-83.html).  Again it was certified bad guys stealing the data.  But because RockYou didn’t take reasonable security precautions to protect that data, they are now facing a very expensive suit and all the negative publicly that that entails.  I am sure that RockYou didn’t want to get profiled by CNET for this reason.

Beyond the civil suits, there is the potential of criminal action.  Just ask HealthNet and Wentworth-Douglass Hospital.  Both companies have suffered data breaches that have resulted in investigations of  by their state’s attorney general office (See here and here). 

The bottom line is that no company should expect sympathy if data in their care gets breached.  Consumers, plaintiffs, and regulatory agencies are just as likely to blame your company as they are the bad guys.   You’re the victim of the data theft, but unless your company has taken all the available precautions it can, you’ll also be viewed as one of the “bad guys”

Posted by Barbara Rogan on January 07, 2010 in Legal Nerd , Log Management & Intelligence , Security | Permalink | Comments (0)

Verizon Thinks You'll Evolve

By Andy Morris, Log Fan

I read Dimitri's take on the Verizon Top 10 Security Predictions for 2010 and thought I'd take a swing at it myself.

Verizon’s security predictions for 2010 are interesting partly because of their insightfulness, and partly due to their lack of insight. You can read their full list of predictions at here, but if you’ll allow me, let me play scrooge.

1) Services will protect themselves.
No they won’t. What most services will do, is appear to protect themselves. They’ll respond to a few highly publicized events with new user interface options that people won’t use properly, and will give the fake appearance of positive change.

2) Malware will not evolve.
This seems about right. Why go to all that fuss and expense of evolving, when most networks still aren’t protected against threats that were discovered ages ago? Mass outbreaks, of course, are for show-off-bored-kids; these days the real money is “on the fringes”. You know, like the Russian Mafia exploiting high street banks for millions. So, no real concern there then. Except that we’re in a recession, and it’s our money they’re stealing.

3) Consumers are getting smarter.
This is possibly the most dangerous of all the predictions. I don’t know if it will be true or false, but as security experts we have to assume it’s false, and build a world that protects the naive, the innocent, the gullible, and that chap that runs with scissors.

4) Windows 7 will be more robust than expected.
Well that’s a low bar - remember Windows 7 was launched on Oct 22, and exploits started turning up as far back as April, but Verizon is right to turn the focus on ISV’s. After all, hackers are after money, and that’s buried in data, and that’s handled by ISV software.

5) Serious finger pointing will occur – criminals think twice.
Yes and no. Finger pointing will occur, but criminals will just shrug. Maybe this is a good time to have a debate about Capital Punishment deterring murderers?

6) Breaches will increase.
Yes they will. The lust for money is powerful motivator.

7.) Nothing happens to non-PCs 8.) CaaS works 9.) Virtualization is not attacked 10.) China will be blamed for everything.
Lets hope so :: I don’t care :: More hoping :: Seems fair.

What does LogLogic predict for 2010? Regardless of whether, all, some, or none, of Verizon’s predictions come true, networks will still be left vulnerable, applications will be un-patched, user error will causes breaches in protocol, and criminals will successfully knock down walls.

But not on a LogLogic protected infrastructure.

We can prevent, capture and prove compliance for whatever 2010 throws at your systems.
LogLogic customers are predicting a stress free, safe 2010.
(No lead paint was used in the making of this post – no thanks to China. Or Nigeria. Or Eastern Europe.)

Posted by Andy Morris on January 06, 2010 in Security , Top10 | Permalink | Comments (0)

Top 10 Security Predictions for 2010

Verizon Security recently posted a set of 10 predictions for 2010 on their security blog. I have my own opinions about their predictions as you'll read below.

To see Verizon’s original predictions, click here:2010 Security Predictions

Our friends at Verizon Security feel that services like Facebook, Google, Twitter, and TinyURL will work to get better controls in place regarding criminal content. They believe that their business model is at stake if they don’t attempt to flag or eradicate nefarious activity... advertisers will start pulling their dinero. And my response to that is "of course they will!" It's an obvious statement. The online services will absolutely do more to try to curb illegal behavior. If they don’t do it, who will?

The recent FaceBook"apps" scandal has made everyone scratch their heads and realize that they're allowing a number of different programs to have access to their accounts and with that, some level of personal information. Twitter has been hacked over and over again. MySpace has vulnerabilities left/right and center. So to say that services will protect themselves is obvious. Whether these hacks or illicit behavior take place to them or on their networks is a variable. It all depends on the vulnerabilities discovered. The web after all is Swiss cheese. Admitting that is the first step.

Our friends at Verizon also feel that Malware will not evolve this year, that Botnets will stay the same as a whole, and there won't be any mass outbreaks or targeted attacks. Personally, I don’t see evolution as necessary when the same ole vulnerabilities still exist. Security best practices weren't followed until specific verticals created requirements to do so. The result was PCI, HIPAA, SOX, ISO17799, and more pop up every day. If businesses would stop thinking of security as an outflow of cash, and instead think of it as a necessary cost of doing business, we'd all be a whole lot safer. The outbreaks will happen when yet another bored 14-year-old finds a vulnerability and decides he’s going to be the next big thing. And chances are, he’ll be rewarded with a big security job somewhere. Funny how that works.

The security team at Verizon also feel that consumers are getting smarter. The impression that there are fewer newbies on the internet, and services are more secure, and that people are generally more aware might be true. In one respect, however, I wholeheartedly disagree. As P.T. Barnum once articulately stated, "A sucker is born every minute." This hasn't changed. Sure, people aren't responding to instant messages on AOL asking for usernames and passwords, but the phishing sites are getting better, the vulnerabilities are becoming more public and people are still falling victim. Think back to the days of "Don't open executables!" which became "Don't open .SCR files!" followed by "Don't open macros!" and then the ActiveX nonsense for malware. At the end of the day, although the public is getting a wee bit wiser, the trojan writers are getting better-er. Claiming that people are more intelligent because your friends haven't been scammed in a while says little about the state of public affairs.

Number four on Verizon’s list states that Windows7 (not necessarily IE8) will prove to be more robust than anticipated (vs. Vista), and that applications are the new targets. These are two completely different statements, and I’m not sure why they ended up in the same paragraph together.

First off, I should warn you – take what I’m about to say with a grain of salt as I am a world-class Windows hater. I will do my best not to let my absolute loathing of all things Microsoft seep out. Oh well. So much for that.

Windows7 is more robust than Vista, but that's not saying much. It’s like saying a 2009 Honda Civic is more robust than a 2008 Honda Civic just because there's new standard leather trim. It's still a Honda Civic. It's still the same car. It’s just dressed up prettier. Windows fans will go on and on about this-and-that device support and stability. We’ll all stay tuned for that one.

Attacking applications as the next step is fairly obvious. Of course crooks are going to go for applications. Applications aren't written to be secure. Writing for security is much more time consuming and therefore more expensive. Coding for security has to be the next evolution in application development. Write for security as the first step. Make security the high priority. Don't write the app, then go back to see if it's secure. This is what causes world class /fail.

Number five on Verizon’s list of 2010 predictions is that government and non-tech organizations worldwide will become increasingly frustrated over SMTP, DNS and SPAM, and they’ll find phishing more and more difficult to thwart. They believe that Microsoft’s legal efforts to can-that-spam, along with a high-profile arrest will somehow cause all the other SPAMMERS in the world to shake in their boots and think twice about their line of work.

*yawn*

Spammers are nothing more than ticks on the backside of the internet. They exist. They suck off their hosts. And then they fall off. If we want to end SPAM tomorrow we have to make the punishment for spamming so severe that the mere thought of it will make these hoodlums shake in fear. Follow the money. Who is profiting? Is it the manufacturer of said product? Is it a reseller? Follow the money. Then once you get them, go after the people who actually BOUGHT something due to a SPAM email. The only reason spammers still SPAM is because someone is buying. Those people should be prosecuted for even responding to SPAM.

Verizon Security also believes that breaches will increase, but on a smaller scale with fewer records compromised. They feel that more money theft will take place with account staff credentials being compromised. And they also believe mid-size businesses will be hit with some sort of compliance mandate to force them to do the right thing. Where Verizon and I disagree is that I see this going in the opposite direction. I see more breaches, more records compromised, more insider threats, more phishers, and more crooks using Western Union to transfer money. 

What I'd love to see is a better than best practices compliance mandate to supersede all mandates. From small business to large enterprise, make everyone play by the same rules regardless of vertical, regardless of industry, regardless of income. One compliance mandate to rule them all. That compliance mandate should not only represent best practices, but step it up a few levels. 

Also, if there was blanket worldwide legal policy that applied to ALL cyber-crooks globally, these scoundrels would no longer go unpunished. A couple of thousand dollars stolen from an account in the U.S. goes a LONG way in some other countries, and not only is it relatively easy to commit these crimes, but there are really no legal deterrents in place to discourage these high tech pickpockets in other countries. Hoodlums can make millions (yes, millions) without any fear of prosecution, and the temptation to pick such low-hanging (albeit forbidden) fruit is very difficult to resist. Let's get downright hardcore on the legal front. Let’s take down these wrongdoers.

Verizon Security went out on a limb when they stated that nothing of note is going to happen to phones, PDA’s, and Macs. Really? Uh…no. Just two weeks ago we all learned about a sneaky little trick to invade unlocked iPhones who have SSH enabled with default passwords. This is just step one. If you look at how many iPhones are on the market, you can see the huge motivation for delinquents to act-a-fool. I see the mobile phone market getting its fair share of security issues.

Although I think Verizon Security has a high level view of what takes place on the side of security, it seems some of the predictions are off in left field somewhere.

One prediction I believe nobody will dispute though, is that 2010 will be a very exciting year in security. And if we're lucky, a few people will realize they need log management to keep an eye on the security of their infrastructure. Stay tuned.

Posted by Bill Roth on December 28, 2009 in Security , Top10 | Permalink | Comments (0)

Citibank, Cyber-Goons and SEM

By Lex van den Berghe, LogLogic Customer Evangelist

The Wall Street Journal today broke news with a story detailing an FBI probe into the possible theft of tens of millions of dollars from Citigroup by a Russian gang of cyber-crooks. But what strikes me as odd and controversial isn’t the theft itself or even the growing trend of this kind of crime, but that Citibank and the "government source" are at odds.

What gives? Are we looking at a bit of irresponsible, shoot-from-the-hip reporting by the Wall Street Journal or something else? This story is clearly a big deal – I mean, we’re talking about *tens of millions* of dollars…and the FBI has allegedly gotten involved.

There’s no denying that priority and urgency continues to escalate as cyber-crime transitions from science fiction to hard reality and cyber-crime has become top-of-mind with consumers of all demographics.

According to the WSJ story, the Citibank attack was initially detected over the summer, but reports seem to indicate that the attack may have actually occurred a year earlier. So, how is it that all that cash went <poof!> and we haven’t heard about it until now. Or even stranger, what’s behind Citigroup’s claim that the thefts never occurred and the WSJ’s report is not true. Joe Petro, managing director of Citigroup's Security and Investigative services, said, "We had no breach of the system and there were no losses, no customer losses, no bank losses." He added later: "Any allegation that the FBI is working a case at Citigroup involving tens of millions of losses is just not true." One important thing to note is that Mr. Petro is not in PR, but rather part of Citi’s security arm. This gives his assertions more credibility. (Sorry PR folks).

I’m no conspiracy theorist by nature, but something definitely smells fishy here.

Folks…the truth is out there. And finding it ain’t rocket science. LogLogic’s log management and security event management tools literally record everything as it happens in even the most complex IT environment, leaving a convenient breadcrumb trail behind that anyone can follow. This breadcrumb trail includes every key stroke, file movement, login, breach, etc…like DNA left behind at the scene of a crime. Deploying these tools in your business IT environment is equivalent to installing one of those black boxes, or flight recorders that they put in every airplane.

As a consumer, I’m always relieved to hear that institutions like Citi bear the burden of absorbing financial losses resulting from these sorts of cyber-crimes, and those of us whose accounts have been cleaned out, usually do get our money back. But that’s not enough. I want these cyber-scumbags to pay for their crimes and more important, I want future cybercriminals to think twice before they choose the dark path. If every institution out there that we trust to guard our money or personal information start using the right tools to safeguard these commodities, things might be a bit different.

Posted by Lex Van den Berghe on December 22, 2009 in Security | Permalink | Comments (0)

Is the United Kingdom at Risk of Korean Cyberattacks due to Government inaction?

Last month, hot on the heels of the US, the UK government published three documents outlining its strategy and position on “cybersecurity”. The Cabinet Office published the “Cyber Security Strategy” and the “Security for the next generation” documents and the Department for culture, media and sport published the long awaited “Digital Britain” report.

Perhaps unsurprisingly, considering this is the first time Cybersecurity has been discussed at any great length if at all, I found the reports quite vague and strategy focussed rather than actually giving out any specific advice or regulation. Clearly these initiatives are a start but as the Korean cyber attacks of the past week point out clearly: foreign governments and criminals are many steps ahead.

I also came away thinking about how the UK documents position shared-responsibility amongst IT departments. In my experience many areas of IT operate in silo’s which results in a lack of important information exchange. These reports don’t go far enough in terms of defining responsibilities and raising awareness of who, how and what should be communicated. It’s all a bit woolly.

The US cybersecurity directives imply businesses should set a baseline for all business from a security perspective – is the UK behind in its approach and weak by not setting out a clear mandate? The recommendations for security awareness and individual responsibility are pretty basic and undefined. It does make my cynical side come out and ponder whether these reports are sincere or just fodder to distract the media from the enormous number of UK government data leaks! They need to take the bull by the horns and get their own house in order. Make security actionable and lead by example - by far the best approach.

With any security or information protection law both the US and UK governments face an uphill battle because of a strong privacy movement.  Protecting the safety of the public at large could invariably compromise the privacy of few – it’s a trade off but just how far will IT go to protect corporate data?  I recently wrote about this in an article for Computer Weekly (see How far should IT managers go to protect corporate data?) – take a read and let me know your thoughts.

Posted by Dominique Levin on July 10, 2009 in Security | Permalink | Comments (0)

Instant Poll: What works best when justifying security spending in your organization?

You are only ONE CLICK away from seeing what your peers are saying. 

Click on one of the choices below to see INSTANT results.

Posted by Dominique Levin on June 30, 2009 in Log Management & Intelligence , Security | Permalink | Comments (0)

LogLogic Buys Exaprotect: 3 Reasons Why Customers Win.

By Dominique Levin
VP Marketing & Strategy

Last week LogLogic announced its intend to acquire Exaprotect. In February we had already announced a partnership with Exaprotect to deliver the LogLogic Security Event Manager. In February we also announced LogLogic Compliance Manager, which has since shipped to the general public, and LogLogic Database Security Manager, generally available later this quarter. Now we have added the Exaprotect Change Manager product line. In a mere couple of months LogLogic went from a singularly focused company with leading log management platforms to having five product lines working together to form the most complete security management suite.

So how does this all benefit customers?  The combined product portfolio answers 3 simple questions for customers:

What is happening?

What is important?

What to do about it?

1. What is happening? Log Management and Database Activity Monitoring.

It all starts and ends with log data. You cannot secure or manage what you cannot see. Therefore, first focus on building a central repository of user and system activity. You do this through aggregating, summarizing and archiving log data. Log data can tell you who are accessing your network, systems and even who are seeing, changing or moving individual information objects. Per a recent SANS survey, 99 percent of customers are collecting (or planning to collect in the next year) some log data but for many it is work in progress. Virtually all collect network data (“who is accessing my network?”) and most collect system-level data (“who is accessing my systems?”). For most companies even collecting a complete activity record remains a work in progress. Leading-edge organizations are now turning their attention to understanding activities around business applications, transactions and monitoring access to specific sensitive information objects. This is particularly true for structured information in databases. Databases are a one-stop shop for valuable data. Organized criminals are targeting sensitive data in databases to sell for $300 per record. Since the data is structured, you know where it resides and you can monitor access to these specific records. LogLogic expanded into database activity monitoring with a specialized database sensor. The sensor sees more than you would through native logs, including activities that are triggered by stored procedures, obfuscated queries and such. This is great as a stand alone product, but at the end of the day, database activity should be analyzed in context with all other activity data – hence the convergence of log management and database activity monitoring.

2. What is important? Compliance management and security event management.

Just having the data on a pile is of course not enough. Once you have a central record of activity, you need look at this information. Few organizations are proactive about this. LogLogic compliance management and security event management applications can help. LogLogic Compliance Manager is about deciding who should be looking at what log data when and then enforcing such log review process through software. Compliance is a collaborative process and Compliance Manager facilitates collaboration on pro-active security. It productizes best practices, presents reviewers with an easy in-box of log review tasks and the ability to annotate and score activities. Ultimately the log review scores roll up into a dashboard that presents executives with the overall timeliness of review and a compliance score. It is still human beings who do the bulk of the actual analysis. LogLogic Security Event Manager goes one step further and uses cross-device correlation and contextual analysis with vulnerability and asset data to prioritize suspicious activities automatically. For example, access to a HR database followed by a large e-mail sent, could be suspicious and needs to be investigated immediately.

3. What to do about it? Change management and database security.

Contextual analysis of log data is cool and it can go a long way turning raw log data into actionable information and even into recommendations. However, security Nirvana would be self healing. Increasingly software could make automated recommendations and predictions about unusual and suspicious activities and could prevent bad things from happening in the first place. LogLogic Change Manager and the LogLogic Database Security agent both have the ability to enforce security policies. Most customers aren’t quite ready to automatically re-configure a firewall policy based on a security alert, but at some point in the future as predictions become more accurate, automatic remediation will become a reality. One area where automated prevention is a reality is in database security. About 20% of database security customers also turn on active blocking. It makes sense that blocking would be more prevalent with systems that can do fine-grain monitoring. It is tricky to kick somebody off the network wholesale based on a security alert. There are still too many false positives. If you get it wrong you seriously hurt productivity. That is not a good thing ever, but especially not in an economic downturn. Most organizations prioritize productivity over security. It is much more acceptable however, to block access to a specific piece of information based on suspicious activity.

In summary, with the addition of Exaprotect, LogLogic can better protection information at a lower cost. This is good news at a time that few customers can afford to maintain the staff and budgets to integrate many disparate point products.  Unified security management also leads to better information protection. Pro-active security monitoring (LogLogic Security Event Manager and LogLogic Compliance Manager), combined with fine grain monitoring (LogLogic Database Security Manager) leads to more accurate prevention (LogLogic Change Manager and LogLogic Database Security Manager) and better information protection.

Posted by Dominique Levin on April 27, 2009 in Compliance , Log Management & Intelligence , LogLogic News , Security | Permalink | Comments (0)

Is your data protected if the company closes its doors?

There has been a lot of buzz in the press lately about what happens to confidential data when employees are laid off and, or an even worse situation, a company goes out of business. This brings up an important issue for enterprises who face stringent industry and government requirements for controlling and monitoring what happens to their information assets. Who is accountable for the corporate data amassed over its lifetime after it closes doors?

One way to keep an eye on your data when you are in business is to automate IT operational tasks for continuous assessment of the risk profile, to bring unusual activity to your attention. Our CEO, Pat Sueltz, discussed the simple and complex aspects of IT process automation with eWeek's editorial director, Michael Vizard, for a podcast this week. You can check out the podcast here.

“Trust but verify” is the mantra in our log management world. Although log management is but one of the defenses in your security armoire, it is an important building block to monitor user and system activities to identify and correct the gap between your security policies and the reality on the corporate network a.k.a the ground. Automated log management can also be extended for security information and event management, database activity monitoring and managing compliance workflows.

If you're looking for other ways to make sure customer and business data isn't literally walking out the door, consider checking out Network World's podcast, "Why ex-employees are stealing your data." The podcast discusses results from a recent study conducted by the Ponemon Institute, "Jobs at Risk = Data at Risk." According to the survey of 945 people who lost their jobs in the past 12 months, 59% admitted to stealing company data and 67% used their former company's confidential information to leverage a new job. A particularly interesting finding we noticed: only 37% of these individuals were actually asked to leave their jobs – the other two-thirds either found a new job or left in anticipation of lay-offs.

In other words, whether employees have ill will against a company or not, there is no excuse for not protecting and monitoring your data. Another article on the subject in CIO Magazine yesterday brings up the issues of private data being auctioned off in fire sales as companies go out of business and improperly disposes of their sensitive corporate and customer information.

The privacy policies that are communicated to external parties like customers, employees, and other partners almost always discuss it in the context of their business as a going concern. They even discuss how the information will be treated in the event of an acquisition or with their other subsidiaries. I have rarely seen them discuss what they do to your information if they just simply shutdown.

Data governance continues to be a challenge today. Disaster Recovery and Business Continuity plans are also more the norm than the exception today. But, if you are the IT Security team, have you given thought to how you would handle the corporate data (including all kinds of PII and logs) in the event of a company closure? As a consumer do we have any rights to the data and content we shared with this company during its life time? Shouldn’t it be just as natural for the company to be responsible here? Just as we plan our legacy in the event of a sudden life changing event in our lives …like wills and trusts?

Posted by on February 26, 2009 in Security | Permalink

Database Security

By Dimitri McKay

If I was to hack your network, what would be my goal? Let’s say, for instance, that I get through your firewall. Maybe I exploit some buffer in your IOS to get through one of your firewalls. Maybe I come in over your VPN. Maybe I make my way into your office under the guise of being a “contractor”. What’s my goal? What’s my target? What is the treasure am I looking for?

In 1950 the FBI’s most wanted bank robber Willie Sutton was arrested. When asked why he robbed banks, Sutton simply replied, "Because that's where the money is.” - and I would relate that back to network security. The databases are where the money is.

I’ve consistently come across companies with a single focus in mind. “The perimeter”. They place so much emphasis on this single portion of security that they lose sight of the big picture. The big picture is of course, A good security system starts with a well-structured security policy. That policy needs to handle all aspects of security, from physical, to operational.

Too often system administrators are left to their own accord, managing the security of their systems with little or no oversight by a higher security administrator. This raises the following questions:

Who ensures system administrators are following security guide lines?
How does an organization ensure all system administrators are applying the latest patches?
What organization ensures that the latest patches have been tested to ensure they do not cause additional system faults?
Who performs security audits on the corporation as a whole?

Here’s some staggering statistics:
In November/08 - 338,000 records (Name/ DOB/ SSN) were stolen from University of Florida
In January/08 - 225,000 records (Name/ SSN/ Account numbers and balances) were stolen from the Davidson Companies.
In July/07 - 8.5 million records (customer data/ bank account and credit card information) were stolen from Fidelity National Information Services.

Where does private data live? Chances are, it’s one or more databases on the network. Databases hold PII (Personally Identifiable Information) which includes customer or employee names, their address, maybe their social security numbers and credit card info. If so, that’s the jackpot!

But how do you even know if you’ve been hacked? If I came over the VPN, would you know? If I breached your PII database, would you know? 3.5% of companies do NOTHING for database security.

For the average company, those who are security focused regarding their databases install a patch between 3 and 12 months after it’s released. For the rest of companies, patches are installed between 12 months and never.

Frankly, this scares me half to death. As a victim of identity theft (who went through the grueling process of fixing my post-credit-apocolypse life, which took years) it makes me sick that databases containing PII are the last thing people are even considering to log or patch or secure.

Most often, the push back from DBA’s is performance. Full logging using the default logging tools from any of your top tier DB’s is going to affect its performance. But there are other options. Options which offer database security without the performance hit.

If I was to hack your network your databases would be my goal. Whether I come in over the VPN, straight through your firewall, or under the guise of being a “contractor”, these databases need to be locked down. That should include, but not be limited to a segregation of the databases from the general public, the authentication of said databases being logged and maintained, and patches periodically installed when released. Not two years later. Databases are the treasure in your castle. They need to be protected.

Posted by on February 17, 2009 in Security | Permalink | Comments (0)

Logging Poll #9 Analysis: Log Protection and Security

This is the analysis of my last poll; the responses are here and also below.

First, the most obvious conclusion: people still don't care much about log security; I am saying that since this was BY FAR the least popular of my polls. Only 24 people responded, so everything below is pretty unscientific.  A good way to explain it: look at the recent media? Do these people care about their key business data and their customer data security? Nope. So, how do we make them care about securing their log data?

Second,  it is entirely unsurprising that 83% of respondents want "Authenticated access to log server." In fact, I'd opine that 100% of people want authenticated access to any of their servers :-) But, this was my "red herring" to set the baseline for the rest of the questions... 

However, this is where the buck stops: other security measures are notably less popular.

Third, "Logging all access to logs" is my favorite and I am happy to see it reported as popular. But do people really do it?  Do you log access to log server OR log access to actual logs? Think about it... I suspect a lot of people who do the latter still answered "yes" to this question. So, does your system allow you to log all access to log data? If not, get LogLogic! :-)

Fourth,  "Reliable / acknowledged network transfer of log data" and "Encryption of log data in transit " are two true "no-brainer" security features; they took the next spot at 45% and 50% of those who answered. They are simple, they are easy, they make  sense - and, obviously, they don't make logs entirely secure so you need to do more. Why only 50%? Where is THE OTHER 50%?!

Fifth, "all things crypto" are below 40%. "Cryptographic hashing of stored logs", "Cryptographic signing of stored log data" and "Encryption of stored log data" all hover at around 30%. I attribute them to general disregard of log security AND reliance on "system security" (separate server, etc) over "data security" measures for log protection.

Finally, I am embarrassed to admit that I missed  the obvious security measure: "Separate server for logging, not accessible from the Internet." Happily, one of my readers added this using "Other security measures" choice. Indeed, this is a good point - and a good idea to do it. Another option mentioned there was "Destroy old logs."  Indeed, it helps log security by preventing unauthorized access to logs.

Possibly related posts:

• Top 11 Reasons to Secure and Protect Logs
• All other polls and their analysis

Posted by on September 05, 2008 in Innovation , Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

Anton Logging Tip of the Day #16: Virtually There - Journey Into VMWare ESX Log Analysis

Following the new "tradition" of posting tips of the week, I decided to follow along and join the initiative.

So, after a long delay, Anton Logging Tip of the Day #16: Virtually There - Journey Into VMWare ESX Log Analysis

CISecurity guide for VMWare (here) and DISA STIG for virtual machines (here) both mandate collection and analysis of VM platform logs; none goes into enough details on what to look for in logs. Let's try to shed some light on security-focused log analysis of VMWare ESX v. 3.x logs.

First, at least until ESXi becomes the default choice, one needs to keep in mind that ESX has "Linux-inside" and thus diving into /var/log will not reveal any "alien technology" (well, not much of it  :-)). However, one of the most useful logs is /var/log/hostd.N which is not a descendant of Linux standard logs. Extensive VM event records are written into this file.

Let's focus on various types of logins to the ESX platform and identify logs that indicate a successful and failed attempts to log in. Here are a few useful examples to analyze:

Successful logins:

• May 30 09:20:42 esx2 su(pam_unix)[9405]: session opened for user root by jhonny(uid=1626)

This is a classic Linux root login message; you can watch for these by searching VMWare ESX logs for "session AND opened AND user AND root."  Notice the user name of the user who switched to root.

• May 30 09:20:34 esx2 sshd(pam_unix)[9364]: session opened for user jhonny by (uid=0)

This is also a classic Linux message for a normal (non-root) user login.

• [2008-05-25 06:57:48.774 'ha-eventmgr' 111639472 info] Event 40645 : User jhonny@1.1.1.1 logged in

This is a VMWare -specific application login to ESX. You can track such events by username, by event ID or by keywords "event AND logged AND user" (if you are using search)

Failed logins:

• May 30 09:20:31 esx2 sshd[9356]: Failed password for jhonny from 1.1.1.1 port 54773 ssh2

Another classic Linux message from the ESX system; a failure to login due to incorrect password.

• May 27 12:06:59 esx2 sshd[4756]: Failed password for illegal user jonny from 1.1.1.1 port 30594 ssh2

A message indicating a failure to login due to incorrect username (note a typo).

• May 25 07:03:48 esx1 sudo:     jhonny : 3 incorrect password attempts ; TTY=pts/0 ; PWD=/var/log ; USER=root ; COMMAND=/bin/bash

This ESX Linux platform message should also be familiar to Linux/Unix admins: it indicates multiple sudo password failures; look for such messages in the logs.

BTW, do you need to be reminded to track NOT only failed, but also successful login events?! This applies to virtual as well as physical environments.

Overall, you must prepare for the future by learning to analyze  VMWare logs, just like you handled "legacy OS", such as Linux/Unix and Windows.

As I said before, I am tagging all the tips on my del.icio.us feed; here is the link: All Security Tips of the Day.

Posted by on August 27, 2008 in Innovation , Log Management & Intelligence , Security | Permalink | Comments (0)

Challenges of Enterprise Cloud Computing

[ Originally posted at OnSaaS ]

Today, the major use of cloud computing for enterprises are still in its infancy (heck the whole cloud computing space is in its infancy). Most enterprises use cloud computing for testing, development and other peripheral tasks. However, most, if any, are using the clouds for production use. This is fairly similar to the virtualization space, where early use of the virtualization technology are for testing and development. Ten years later, we are seeing more and more enterprises adopt virtualization for production use and virtualization has become main stream.

What are these challenges for enterprise cloud computing? I have tried to summarize them here (in no particular order).

Data Governance

I've written extensively about the need for data governance in previous posts. In essence, enterprises have a ton of sensitive data that requires access monitoring and protection. Data (and information generated from the data) is the life blood of many enterprises, the loss of control will not be acceptable. Whole markets (read: DLP) are created to protect the enterprise data and information. On top of all that, enterprises must comply with many of the regulations that require data governance. By moving the data into the cloud, enterprise, for now, will lose some capabilities to govern their own data set. They would have to rely on the service providers to guarantee the safety of their data.

I hate to invoke the ILM acronym but much of data governance is about

• Creation and Receipt
  
• Distribution
  
• Use
  
• Maintenance
  
• Disposition
  

So who's tackling this problem? As far as I know, nobody is and nobody really can except for the service providers themselves. It is really up to the service providers such as Amazon, Google and Salesforce to provide guarantees that customer data are safe and access to data are restricted and protected.

Manageability

There are some great IaaS/PaaS out there, including Amazon's web services (S3, EC2, EBS, etc), Google's App Engine, Salesforce's Force.com, Joyent, etc. However, most of these are raw infrastructures and platforms that do not have great management capabilities. This is not unusual. Throughout computing history, raw capabilities will generally appear on the market first, then management of these raw capabilities become a differentiator when competition heats up. Just look at the blade server and virtualization spaces as these are great examples of that trend. The hypervisor was the key technology that enabled enterprise virtualization; however, that piece is now being given away (see VMware's ESXi) and management capabilities becomes the main differentiator.

Cloud computing is no different. An example of missing management capabilities for cloud infrastructures is auto-scaling. Amazon EC2 claims to be elastic; however, it really means that it has the potential to be elastic. Amazon EC2 will not automatically scale your application as your server becomes heavily loaded. It is still up to the developer to manage that scalability problem.

So who's tackling this problem? Many startups have recognized the need for management early on and have built management capabilities on top of the existing cloud infrastructure/platforms. RightScale is one of the early pioneers in this space. Their solution solves many of the management issues such as auto-scaling and load balancing.

Monitoring

Monitoring, whether is for performance or availability, is critical to any IT shop. We are not talking about just how much CPU or memory the machines are using. We are talking about performance of transactions and disk IO and others. CPU and memory usage are misleading most of the time in virtual environments. The only real measurement is how long your transactions are taking and how much latency there are. According to High Availability's article on latency:

Amazon found every 100ms of latency cost them 1% in sales. Google found an extra .5 seconds in search page generation time dropped traffic by 20%. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition.

So who's tackling this problem? Hypernic's CloudStatus is one of the first to recognize this issue and developed a solution for it. They started with monitoring of Amazon's web services, then recently added monitoring for Google App Engine. In addition, RightScale's solution can also provide monitoring for the virtual machines under their management.

Reliability and Availability

I won't beat the dead "Gmail down, EC2 down, etc down" horse here. But the truth of the matter is enterprises today cannot reasonably rely on the cloud infrastructures/platforms to run their business. There’s almost no SLAs provided by the cloud providers today. Even Jeff Barr from Amazon said that AWS only provides SLA for their S3 service. I haven’t researched the SLA issue so not sure how true that is. But if it’s true, I think this will be one of the biggest factor, if not the biggest factor, in enterprise adoption. Can you imagine enterprises signing up cloud computing contracts without SLAs clearly defined? It’s like going to host their business critical infrastructure in a data center that doesn’t have clearly defined SLA.

We all know that SLAs really doesn’t buy you much. In most cases, enterprises get refunded for the amount of time that the network was down. No SLA will cover business loss. However, as one of the CSOs I met said, it’s about risk transfer. As long as there’s a defined SLA on paper, when the network/site goes down, they can go after somebody. If there’s no SLA, it will be the CIO/CSO’s head that’s on the chopping block.

So who's tackling this problem? Well, again, no one is today as far as I know. Maybe some startup will come up with clever idea to provide SLA as a third party vendor (read: cloud insurance.) Or maybe the cloud providers will grow/wake up and actually do something to encourage the enterprise adoption.

Virtualization Security

Security is a huge area that encompasses many different things, including the standard enterprise security policies on access control, activity monitoring, patch management, etc. On top of that, virtualization security is something that most enterprises are just starting to grasp but don't fully understand. Many IT people still believe that the hypervisor and virtual machines are safe. Recent presentations from Blackhat has demonstrate that we shouldn't sleep so tight at night. As IT shops get more educated on the virtualization security issues, it will become one of the factors they will consider when they move into the cloud. Access control and monitoring of the virtual infrastructure will be on top of their mind.

So who's tackling this problem? There are quite a few startups like Reflex, Blue Lane and Catbird that are creating privileged VAs that claim to protect the VAs running on VMware's ESX servers. However, ensure you do your research on the performance of these solutions first before adopting one of them. Other startups (unnamed) are creating interesting solutions in protecting the actual virtual infrastructure themselves, e.g., how do you protect and monitor access to the ESX servers? how do you control and monitor the movement of virtual machines using live migration or VMotion.

---

Cloud computing is here to stay. It will be the next big wave and will be adopted by enterprises. However, the industry as a whole needs to answer some of these challenges and ease the enterprises' concerns.

Posted by on August 23, 2008 in Cloud Computing , Security | Permalink | Comments (0)

Tough Security Questions for SaaS Providers - Part 2

[ Originally posted at OnSaaS ]

This is part 2 of the tough security questions for SaaS providers. In part 1 of the series, we asked the following questions:

1. Data Locality - Where's my data?
2. Data Segregation - How is my data segregated with other customers, potentially my competitors?
3. Data Access - Who can access my data in your company?
4. Access Audit - Who has accessed my data and where's my access logs?

We are continuing this discussion with the following questions in part 2.

5. How are the users authenticated and authorized?
6. Web Application Security - How secure is the SaaS provider's web application?
7. Data Breaches - How do you protect my data from insider breaches?
8. PCI DSS - Are you compliant with PCI DSS?

5. How are the users authenticated and authorized?

Companies have spent hundreds of man years and millions of dollars trying to setup single-sign-on systems inside the corporate firewalls. Most companies, if not all, are storing their employee information in some type of LDAP servers. In the case of SMB companies, a segment that has the highest SaaS adoption rate, Active Directory seems to be the most popular tool for managing users. In many cases, companies have designed their IT infrastructure so that all authentication, including VPN, web proxy, file server, and others will go through this single infrastructure. The process of employee onboarding and termination is much easier this way.

Just as companies start to have some success, the advent of the SaaS model changes the scenario again. With SaaS, the software is hosted outside of the corporate firewall. Many times user credentials are stored in the SaaS providers' databases and not part of the corporate IT infrastructure. This means SaaS customers must remember to remove/disable accounts as employees leave the company and create/enable accounts as come onboard. In essence, having multiple SaaS products will increase IT management overhead.

SaaS customers will start asking questions on identity and access integration and providers would be wise to design such features in early on. For example, SaaS providers can provide delegate the authentication process to the customer's internal LDAP/AD server so that companies can retain control over the management of users.

6. Web Application Security - How secure is the SaaS provider's web application?

One of the "must-have" requirements for a SaaS application is that it has to be used and managed over the web (in a browser.) This creates an interesting scenario. In the on-premise scenario, when a vulnerability is found, at least you have your firewall protecting the application so you may get a bit more time to patch it (assuming the application vendor provides the patch in a timely fashion.) However, in the SaaS world, there is no such luxury. Any vulnerability identified can potentially have detrimental impact on all of the customers. Even leading security companies aren't immune to security holes in their web applications.

Web application security is quite a hot topic these days and it's discussed by many security researchers such as rmogull and RSnake. Here's an interesting article on "What web application security really is".

Verizon Business recently released their Verizon Business 2008 Data Breach Investigations Report. Of all the breaches, 59% of the breaches involve hacking, with the following breakdown:

• Application/Service layer -39%
• OS/Platform layer - 23%
• Exploit known vulnerability -18%
• Exploit unknown vulnerability - 5%
• Use of back door -15%

Attacks targeting applications, software, and services were by far the most common technique, representing 39 percent of all hacking activity leading to data compromise. This follows a trend in recent years of attacks moving up the stack. Far from passé, operating system, platform, and server-level attacks accounted for a sizable portion of breaches. Eighteen percent of hacks exploited a specific known vulnerability while 5 percent exploited unknown vulnerabilities for which a patch was not available at the time of the attack. Evidence of re-entry via backdoors, which enable prolonged access to and control of compromised systems, was found in 15 percent of hacking-related breaches. The attractiveness of this to criminals desiring large quantities of information is obvious.

Currently there's really no mandate or requirement for SaaS providers to provide detailed security analysis of the SaaS application. However, it would be wise for the SaaS providers to start considering something similar to what PCI DSS has required of the merchants:

• 6.5 Develop all web applications based on secure coding guidelines such as the Open Web Application Security Project guidelines. Review custom application code to identify coding vulnerabilities. Cover prevention of common coding vulnerabilities in software development processes, to include the following:
  • 6.5.1 Unvalidated input
  • 6.5.2 Broken access control (for example, malicious use of user IDs)
  • 6.5.3 Broken authentication and session management (use of account credentials and session cookies)
  • 6.5.4 Cross-site scripting (XSS) attacks
  • 6.5.5 Buffer overflows
  • 6.5.6 Injection flaws (for example, structured query language (SQL) injection)
  • 6.5.7 Improper error handling
  • 6.5.8 Insecure storage
  • 6.5.9 Denial of service
  • 6.5.10 Insecure configuration management
• 6.6 Ensure that all web-facing applications are protected against known attacks by applying either of the following methods:
  • Having all custom application code reviewed for common vulnerabilities by an organization that specializes in application security
  • Installing an application layer firewall in front of web-facing applications.

Additional sources of information provided as a starting point for more information on web application security would include

• OWASP Top Ten
• OWASP Countermeasures Reference
• OWASP Application Security FAQ
• Build Security In (Dept. of Homeland Security, National Cyber Security Division)
• Web Application Vulnerability Scanners (National Institute of Standards and Technology)
• Web Application Firewall Evaluation Criteria (Web Application Security Consortium)

Trey Ford of Security Spin Control has a fairly good explanation of the recently released PCI information supplement on requirement 6.6.

SC Magazine also has an article on Deconstructing PCI 6.6 for the management folks.

7. Data Breaches - How do you protect my data from insider breaches?

In the Verizon Business breach report blog, Verizon Business stated that

While criminals more often came from external sources, and insider attacks result in the greatest losses, criminals at, or via partner connections actually represent the greatest risk. This is due to our risk equation: Threat X Impact = Risk

• External criminals pose the greatest threat (73%), but achieve the least impact (30,000 compromised records), resulting in a Psuedo Risk Score of 21,900
• Insiders pose the least threat (18%), and achieve the greatest impact (375,000 compromised records), resulting in a Pseudo Risk Score of 67,500
• Partners are middle in both (73 39% and 187,500), resulting in a Pseudo Risk Score of 73,125

Many SaaS advocates claim that SaaS providers can do a better job at protecting the customers' data. Unfortunately, just because the data is now in the cloud, it does not reduce the risk of insider breaches. Insiders still have access to the data, they are just accessing it a different way. Just because the data is in the cloud, the responsibility of segregation of duties and access authorization still fall on the customers, not the SaaS or cloud computing providers. So yes, it may reduce the chance of insiders getting direct access to, say, a database, it does not in any way reduce the risk of insider breaches. In fact, it may even increase the possibility as you now have to take into consideration of the cloud or SaaS providers’ employees. They have access to a lot more information and a single incident could expose information from many customers.

SaaS providers should be prepared to answer questions on what tools and processes are utilized to ensure segregation of duties and protect from insider breaches. Remember, in the case of the mult-billion dollar insider incident at Société Générale, IT management had implemented all of the controls recommended by auditors, but nobody was monitoring them. So it's extremely critical to be able to show the processes around these security controls.

8. PCI DSS - Are you compliant with PCI DSS?

PCI DSS has a specific section for hosting providers (including SaaS providers):

Requirement A.1: Hosting providers protect cardholder data environment

As referenced in Requirement 12.8, all service providers with access to cardholder data (including hosting providers) must adhere to the PCI DSS. In addition, Requirement 2.4 states that hosting providers must protect each entity’s hosted environment and data. Therefore, hosting providers must give special consideration to the following:

A.1 Protect each entity’s (that is merchant, service provider, or other entity) hosted environment and data, as in A.1.1 through A.1.4:

• A.1.1 Ensure that each entity only has access to own cardholder data environment
  
• A.1.2 Restrict each entity’s access and privileges to own cardholder data environment only
  
• A.1.3 Ensure logging and audit trails are enabled and unique to each entity’s cardholder data environment and consistent with PCI DSS Requirement 10
  
• A.1.4 Enable processes to provide for timely forensic investigation in the event of a compromise to any hosted merchant or service provider.
  

A hosting provider must fulfill these requirements as well as all other relevant sections of the PCI DSS. Note: Even though a hosting provider may meet these requirements, the compliance of the entity that uses the hosting provider is not necessarily guaranteed. Each entity must comply with the PCI DSS and validate compliance as applicable.

Simply put, SaaS providers must be compliant with PCI DSS in order to host merchants that must comply with PCI DSS.

We will continue with our tough security questions in part 3 of this series.

Posted by on July 07, 2008 in Cloud Computing , Compliance , SaaS , Security | Permalink | Comments (0)

Tough security questions for SaaS providers - Part 1

[ Originally posted at OnSaaS ]

We will be writing a series of blog posts on the tough questions that SaaS providers can expect to get from customers or they should ask themselves. The questions will span many different areas including security, compliance, sales, marketing and operations. This is Part 1 of the security questions.

As we mentioned previously here, one of the biggest obstacle to enterprise SaaS adoption is the issue of trust. Customers are asking SaaS providers "Can I Trust You?!" The security analysts and warriors are asking similar questions.

Some SaaS advocates have argued that concerns for SaaS security are just red herring. It is true that, to date, there hasn't been any major breaches amongst SaaS providers. However, we have already seen some activities such as the breach at Salesforce.com. In addition, we have seen many anecdote evidence that multi-tenant architectures in the B2C (e.g., Flickr, YouTube) world are prone to data leakage.

One may argue that these are consumer sites and are not relevant for the SaaS providers. However, the same technologies and architectures are being used in both the consumer and enterprise world. In fact, as the trend of IT consumerization continues, we will see more and more of the consumer technologies being used in enterprise applications. Think about it this way, what if this Salesforce.com and your customer list popped up in your competitor's screen?

SaaS providers should be prepared to answer security questions from customers and enterprises. Here are a list of questions that SaaS providers will likely get asked during customer trials/evaulations.

1. Data Locality - Where's my data?

Due to compliance and data privacy laws in various countries, locality of data is of utmost importance in many enterprise architecture. For example, in many EU and South America countries, certain types of data cannot leave the country because of potentially sensitive information. In addition to the issue of local laws, there's also the question of whose jurisdiction the data falls under when an investigation occurs. In most cases, the government where the data is housed will likely win. A good example of this type of concern is when the French cabinet banned the use of Blackberry devices.

Many enterprises have architected around these issues for the on-premise software they install. However, with cloud computing and SaaS, this issue is even more exasperated. In a cloud computing environment, sometimes you don't know where your data is stored or where your application is being run; and some proponents of cloud computing are also saying that you shouldn't have to worry about where the computing resources are as long as your application is running and behaving as it should. However, other leaders in the cloud computing space are taking note of the data privacy and locality issues. For example, Amazon recently announced the availability of an European S3 cloud, and Salesforce.com is also planning Singapore data center.  

Given the regulatory compliance and data privacy concerns, SaaS providers should be ready to answer tough questions about where their computing resources are and will customer data be ever transferred outside to another jurisdiction with different laws.

2. ata Segregation - How is my data segregated with other customers, potentially my competitors?

Everyone's talking about the benefits of multi-tenancy in the SaaS world, but many seem to ignore one of the biggest security concerns, mixing customer data together, that came along with multi-tenancy.   

One of the reasons that hampered SaaS adoption initially was trust. End users must trust that the SaaS providers have the best security in place to protect their data and never expose their data to anyone outside of the authorized domain. Therefore, the ability to segregate data by end customer is a critical requirement for the SaaS providers. There are many architectural methods in segregating the end customer data. At the end, the requirements come down to that users must never see the data that they are not authorized to see, and that end customer's data should never be exposed to other end customers. 

Saas Providers would be wise to consider data segregation early on in the architectural design. For most ISVs turning into SaaS providers, this is an unfamiliar territory and should seek guidance if possible. SaaS providers should also understand the customer concerns and address them early on.

3. Data Access - Who can access my data in your company?

Enterprises have spent hundreds of thousands of dollars on identity and access management systems, log management systems and other software to ensure that employees access only information they are allowed. Within the confines of their firewalls, IT organizations may feel that they have the situation somewhat under control. The advent of SaaS have changed that. With the company data outside of the firewall and in a "cloud," IT organizations no longer can control who and when their data-in-the-cloud will be accessed. Without visibility into the cloud, IT organizations are accepting a much bigger risk compare to when everything's inside the firewall. Even though many SaaS providers have offered various capabilities such as authentication integration with customers' own LDAP servers, this perception of lost control is a difficult hurdle to get over.

SaaS providers offering cloud services, whether it's infrastructure, platform or application, should accept the responsibility of protecting customer data as a single breach could affect all of the customers. SaaS providers must be prepared to help customers understand their security policies on user access, activity monitoring as well as segregation of duties.

4. Access Audit - Who has accessed my data and where's my access logs?

The last few years we have seen a rise of log management and SIEM solutions aimed at compliance-aware organizations. These products is responsible for collecting, analyzing, correlating, archiving and reporting on all activities happening inside an IT infrastructure. Part of the reason these products became such a success is because of the need to track and monitor user activities in the world of regulatory compliance. In addition to compliance, IT organizations use logs to help them identify security issues, perform troubleshooting and forensics analysis, and analyze traffic and user patterns.

With software in the cloud, network, system and application logs are no longer easily accessible by IT organizations. They either have to negotiate access to these logs during contract time, or they have find new ways of monitoring user activities. Given that the IT organizations don't "own" the software, it makes it even more difficult to "hack" around the system. Without access logs, IT organizations may not be able to answer simple questions from auditors, such as "who have accessed the financial information in the past quarter?"

Knowing how critical access logs are to compliance, operations and security matters for IT organizations, SaaS providers should consider providing access logs as a part of their normal service or have it as an option for customers. As an example, Amazon's S3 service offers options to enable and download access logs.

We will continue with our tough security questions in part 2 of this series.

Posted by on July 07, 2008 in Cloud Computing , Compliance , SaaS , Security | Permalink | Comments (0)

Anton Logging Tip of the Day #15: Fear and Loathing in Event 560 (and 562 and 567)

Following the new "tradition" of posting a security tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative. One of the bloggers called it "pay it forward" to the community.

So, Anton Logging Tip of the Day #15: Fear and Loathing in Event 560 (and 562 and 567)

This tip digs into a seemingly simple, but really VERY esoteric subject: monitoring file access and modification via a Windows event log. Now, some people - who never studied this subject - tend to have a very simplistic view of this: just enable Object Access auditing, then right-click on a file or directory, click Security->Advanced->Auditing and then pick what types of events will be logged and by what accessing entities (i.e. users or computers). OK, so this will produce some logs, that is for sure. But are they useful?

First, why are we doing this? We typically need to know the following when we audit file access in Windows (or any other OS for that matter) for security (monitoring and investigation) or compliance:

• Time/date
• Computer where it happened
• User who touched the file
• Application he used to access the file
• File name + location (directory, share, etc)
• Type of access (read, write, create, delete, etc)
• Status (i.e. success or failure)

Can we get this from the above logs? No.

What? No!?! Really?

Yes, really. We can get some of the above, some of the time, not all of the above, all of the time. Here is an example, we are looking at event ID 560 (picture) and then at an extract from its description field.

Event:

Description (selected field):

Object Server: Security

Object Type: File

Object Name: C:\0\TestBed\simple_text_file.txt

Image File Name: C:\WINDOWS\system32\notepad.exe

Primary User Name: Anton

Primary Domain: XXXXXX

Accesses: READ_CONTROL

SYNCHRONIZE

ReadData (or ListDirectory)

WriteData (or AddFile)

AppendData (or AddSubdirectory or CreatePipeInstance)

ReadEA

WriteEA

ReadAttributes

WriteAttributes

WTH is that? Well, we know that the user  'Anton' has successfully read? wrote? changed attributes? did something? with a file named "C:\0\TestBed\simple_text_file.txt" using a program named "C:\WINDOWS\system32\notepad.exe." That's the best we can get, in this case! We may try to look at event IDs 562 and 567, but this missing information (i.e. the exact action performed) will not be added.

BTW, there will be  a few more dozen (sometime hundreds!) of the 560s, 562s and 567s  produced - all from just opening the text file in a notepad. The above event is notable for having BOTH "notepad" and "simple_text_file.txt" in the same event; others will have either of the two.

Anything else gets in the way? Yes, lots! MS Office will write to all files, even just opened for reading (with no user modifications to the content whatsoever), which will screw up your log monitoring efforts. If the file is on a share, more information will be missing (e.g. username might be).

So, how to use Windows event logs for file access tracking?

1. Enable logging (as described above)
2. Pick events 560 (most useful) and 562, 567 (useful too)
3. Look for fun filenames that might be touched by the users (have a list of files and users handy)
4. Figure out what programs were used to access them (this is called "Image File Name" in "WinLogSpeak")
5. Ponder the 'Accesses' section of each event until your brain turns blue :-) or until you decide whether such access is authorized or not...

Overall, this is still very useful for file access monitoring, but the process is somewhat painful.

BTW, I am tagging all the tips on my del.icio.us feed. Here is the link: All Security Tips of the Day.

Posted by on May 08, 2008 in Compliance , Innovation , Log Management & Intelligence , Security | Permalink | Comments (0)

From Log Apathy to Log Enlightenment

So, I was talking to this small log management vendor the other day and he confided to me that his product faces fierce competition in his target market (which is, important to note, small to medium companies with 10-100 systems): and this competition is apathy.

More specifically, his prospects either just blow him off by saying "pah, who needs logging!" or they profess their undying love of all things logging - and then still don't buy his product (which is priced, shall we say, "to go")

Admittedly - and somewhat tongue-in-cheek, these are the same companies that form the core of today's botnets (due to various reasons including their scarce resources) and enable RBN to deliver high-quality malicious services to criminal enterprises worldwide. Still, if you happen to have thoughts along the line of "who needs logs?" or "ah, logging? it will come later!", you really deserve a nice fat check from RBN and other malicious "hacking" syndicates since it is extremely likely that your overall attitude towards security is just as misguided...

But how to progress from such ... what was before the Stone Age? ... Sharpened Stick Age? to modernity? Most companies go through the following stages in regards  to their logging:

1. Deep log ignorance: "Logs? What are those?"
2. Shallow log ignorance: "Later...later...later... #37 on the TODO list."
3. Log collection: "We gather and store dead log data...cold."
4. Log searching: "We will dig into the pile when we have to ... hopefully never!"
5. Log analysis and reporting: "We know our logs - and what they mean"

(also see my post "Natural Flow of Log Management" for some specifics)

Of course, compliance (PCI DSS and others) helped move people from 1. and 2. to 3., but, sadly,  people often get stuck at 3. (just collection) or 4. (collection  + maybe search) and never progress to Logging Enlightenment of 5.

Yes, PCI DSS and other regulations mandate not just log collection, not just dead cold log storage, but also log review (daily, in case of PCI DSS Requirement 10), but "review" happens to be the item that gets overlooked  all too often.

Why is that?

I think the reason is that log analysis is still too hard and still not automated enough for an average organization. Yes, I did see some corporations that built their own log analysis systems that - surprise! - exceeded the best available from the vendors [at the time]. However, a typical company IT department would not have Ph.D. poring through hardcore text mining research papers in order to improve their home-grown log analysis AI. They expect the vendors to  eat the logs, chew on them for a bit - and then spit out the answers.

Are we there yet? No, but we will be!!!

Posted by on April 22, 2008 in Compliance , Innovation , Log Management & Intelligence , Security | Permalink | Comments (0)

RSA 2008 Summary

So, RSA 2008 has come and gone.  Now that everybody has recovered from the information overflow, it is time for summaries and reflections. Before we begin, go read my RSA Impressions (Part 1,2,3,4). Next, read what others said about RSA 2008 (via my del.icio.us feed). Then reflect on past RSA shows (2006, 2007).

Ready now?

First, what was the theme? Unlike in the past, I personally couldn't pick any. The candidates were GRC (and compliance + risk management) , DLP (and other data-leak/-theft technologies), IAM (and identity enablement), but none quite measured up to being a show "theme."

What I saw too much off? Even though their numbers have shrunk, I still saw too many NAC vendors there (Lockdown, here we come!). One of my friends joked that there were more "GRC vendors" than NAC vendors, but both were in low enough numbers to make it a trend. As far as loud noises back from RSA 2007, "identity-driven this or that for security" was still very visible. What is also bizarre is that people still start log management companies. I saw a couple of new ones - amazing, isn't it? Yes, it is a hot space, but come on! Accept that LogLogic is the leader and be done with it :-)

Overblown messages? "Information-centricity." It was cool and hot :-) and new when Hoff said it.  But when it trickled down to keynotes of some "trailing edge" vendor executive, it became boring and stale. And, while we talk about "information centricity" people must still worry about "A" (availability) first (see this discussion).

What I didn't see enough of? VOIP security. Somehow this previously hot trend is quieted down. Also, I saw a lot of web application security vendors, but I think that is still not enough as this is an area with a raging fire, not just "some hotness." Also, I expected to see more vendors messaging around (and, actually helping with!) fraud. Dan Geer's Verdasys kinda mentioned that, but pretty much in passing. Is fraud handled outside of security (and thus out of RSA)? I am not sure.

What I didn't see at all? I didn't see much "market consolidation" - no huge deals, no vendors of note "taken out", etc. Still a huge number of security companies around ... some with products that seem deeply out of touch with today's threat landscape.  One of the speakers said that nowadays "no single security pure-play expected to change the world", but it sure seems like many will try!   Along the same line, Mike R said that such shows are 18-24 months ahead of what "normal" people deploy. This might explain the VOIP and other missing items.

As I said before, "consumerization" of IT - i.e. IT infrastructure, servers, laptops, storage, services, computing resources, applications, etc provisioned outside of IT departments was an elephant in the room. It is not simply "unmanaged IT" or "consumer-grade IT for business", it is the whole "not-IT-department IT" phenomenon. Yes, via mashups it even includes "non-IT application development" (read this fun 451Group report on that). Security implications of this are nothing short of ginormous.

In light of this, I liked how one presenter said this: "we lost the desktop" - meaning "1/3 is managed by users [not IT], 1/3 is unmanaged and 1/3 is compromised/ "managed" by hackers."  Sad but true... Dave Aitel used to joke how in the future banks will have to "re-compromise" your PC so that some temporary security can be established for you to transact business with them. Are such horrifying times upon us already? Maybe desktop virtualization will help with this ...

Overall, RSA 2008 show was an enjoyable, enlightening and thought-provoking experience, as usual. I've heard it was the same even for people who didn't attend a single presentation - indeed, RSA is about people, not slides...

See you next year at RSA 2009!

Posted by on April 16, 2008 in Compliance , Innovation , Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

On Trusting Log Timestamps

It is about time: specifically, timing logs. As I said in my Log Trust and Protecting Logs from Admins posts, the issue of trust is critical in the logging world. After all, logs = accountability; and the latter in unthinkable without trust. If we are to at least pretend that logs objectively record events and user actions, we need to unambiguously establish WHAT happened and WHEN. This post deals with the 'WHEN'  issue.

So, can we trust that the time stamp in the log file or the one added by the log management system correctly describes when the recorded event actually happened?

We will start from locating the timestamps in logs. Most of the log formats, such as file-based logs (web servers and proxies, various application, some security gear, etc) and syslog, Windows event logs, database audit tables, etc contain a timestamp. Here are a few examples:

1. Mar 20 10:03:47
2. 19/Mar/2006:04:27:33 -0800
3. Sun Mar 19 04:45:19 2006

In fact, once I saw somebody use a timestamp to define logs as "timed records of IT activity." So, time is critical for logs being, well, logs :-) At this point it is worthwhile to note that file-based logs will contain a timestamp IN the file, while syslog records arriving over the UDP or TCP port 514 connection are usually timestamped upon arrival BY the syslog daemon (using its own "knowledge" of time) - and then it shows up in the syslog files in /var/log.

Let's assess whether this "in-log timestamp" provides an adequate way of timing the actual events that are being logged. Answering this question is important for investigations and troubleshooting, but becomes  nearly a matter of life and death in case of log forensics. Here are some fun cases and issues to consider:

First, what are the chances of a  completely false timestamp in logs (BTW, today is Jan 1, 1970!) When might that happen? Typically when a logging system own clock is reset or not set correctly. This timestamp clearly should NOT be trusted. How do you then time this event? This is a separate story that will be discussed some other time...

Second, it’s always 5PM somewhere: in other words, what time zone are your logs in? EST? PDT? GMT? UTC? Or any of more than 24 other possibilities. If you have no idea, you should not trust the timestamp.

Third, are you in drift? Is your system clock? Those pesky drift seconds turn into minutes that then work to undermine the accuracy of timing the records (and thus your certainly and trust in evidence quality)

Fourth, syslog forwarder mysteries are plenty: some of the syslog messages will be delayed in transit and then  be timestamped by the final recipient daemon, thus completely losing when the event was originally logged. Admittedly, this delayed syslog messages are rare, but as more people employ buffering syslog daemons (e.g. syslog-ng), it might happen more often.

Fifth, more esoteric, but still real (and really annoying!): some system logs will contain two timestamps. If you don't possess in-depth knowledge of this specific log, confusion has a chance to undermine the trust as well (so, which timestamp should I use?)

Sixth, most people will not think that they will fall to something that stupid: 24 vs 12 hour time. However, when facing an unknown (and poorly designed!) log format, beware that 5:17 might well be 17:17...

Finally, if you know that something got logged at 5:17AM, then when did it happen? Beware of "Log lag!" This issue is actually too tricky to give it justice here... The simplest example is when the process leaves a log record when it exits, not when it starts, which might happen days earlier (thus creating a log lag).

As we dive into more issues with timing logs, we also need to think about sequence timing and absolute timing. Sequence of logged events is a critical fact! Miss the sequence and the whole “house of cards” goes …  But! Absolute time is also important! Can we be assured of both being correct all the time? (hint: no)

So,  when you look at logs next time and you see a timestamp there - start thinking about all of this :-)

Posted by on April 04, 2008 in Innovation , Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

Quick and Fun Poll on Behavior Change

While I am analyzing my previous poll, here is a quick and fun one: do we change our behavior when monitored? Vote away!

Posted by on March 27, 2008 in Compliance , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Audit/Monitor Controls or Audit/Monitor BEFORE Control?

Back in 2004, Forrester paper called "The Natural Order Of Security Yields The Greatest Benefits" proclaimed that "the adoption of security has a natural order: 1) authentication; 2) authorization; 3) administration and 4) audit." Note that audit which, in this case, broadly includes audit, monitoring and detection, comes last. It seems to be fairly in line with common sense: you audit the controls after you put them in place; you monitor after you have authentication and authorization taken care of and you detect the violations after you organized your administration.

The paper clarifies: "With people and contexts defined, protective controls in place, and policies outlined, the fourth set of questions [i.e. regarding the audit] includes: “What happened?”, “What is happening?” or, especially, “Is it working?”

However, is this really so? Or, is this always so? First, when reality collided with plans,  many of the organizations that followed that wisdom got mired in one phase (e.g. in trying to get authentication under control) and ended up having no audit whatsoever: in other words, they are flying blind while implementing controls!  Second, in some cases controls (authentication, authorization, administration) will actually be impossible to implement, while audit will be possible. Imagine retrofitting a legacy application for granular authorization? Third, in some cases implementing prevention/control will be much more complicated, compared to implementing audit: thus, people will face a choice between a half-baked control to a full-blown audit capability? An example will be managing which file each user can access vs monitoring/auditing which file each user have accessed. The latter is doable, while the former is next to impossible. Another way to phrase it is "reactive but possible" vs "proactive but impossible" (hint: pick the former :-))

I think the idea of putting audit first in some cases is the way we'd need to progress. "Wow, what a blasphemy!", some might say ...  After all, if you have not defined controls, what are you going to audit? But remember that audit is meant broadly in this context and thus the opposite question is very relevant: what are you going to control if you have no idea what is going on? People sometimes define a security policy based on how things should be (and then WSJ happens - refresh your memory of the "WSJ saga" here), but then spent years trying to bring policy and reality together (and end up with an environment which is "half-controlled") Won't it be better to audit first, then control?  Or, at the very least, to run BOTH project at the same time so that improvements in control are audited and audit discoveries lead to control improvements!

Obviously, "do IDS before IPS" falls under the same principle: monitor first, [try to] control second. Here is another example: implement log management before identity management. Looking at logs will tell you what privileges the users actually use for doing their daily jobs. Then you can mix it up with what the idea access policy will be.

So, think about it! Questioning the common wisdom does often bring interesting insights.

Posted by on March 06, 2008 in Innovation , Log Management & Intelligence , Security | Permalink | Comments (0)

Audit/Monitor Controls or Audit/Monitor BEFORE Control?

Back in 2004, Forrester paper called "The Natural Order Of Security Yields The Greatest Benefits" proclaimed that "the adoption of security has a natural order: 1) authentication; 2) authorization; 3) administration and 4) audit." Note that audit which, in this case, broadly includes audit, monitoring and detection, comes last. It seems to be fairly in line with common sense: you audit the controls after you put them in place; you monitor after you have authentication and authorization taken care of and you detect the violations after you organized your administration.

The paper clarifies: "With people and contexts defined, protective controls in place, and policies outlined, the fourth set of questions [i.e. regarding the audit] includes: “What happened?”, “What is happening?” or, especially, “Is it working?”

However, is this really so? Or, is this always so? First, when reality collided with plans,  many of the organizations that followed that wisdom got mired in one phase (e.g. in trying to get authentication under control) and ended up having no audit whatsoever: in other words, they are flying blind while implementing controls!  Second, in some cases controls (authentication, authorization, administration) will actually be impossible to implement, while audit will be possible. Imagine retrofitting a legacy application for granular authorization? Third, in some cases implementing prevention/control will be much more complicated, compared to implementing audit: thus, people will face a choice between a half-baked control to a full-blown audit capability? An example will be managing which file each user can access vs monitoring/auditing which file each user have accessed. The latter is doable, while the former is next to impossible. Another way to phrase it is "reactive but possible" vs "proactive but impossible" (hint: pick the former :-))

I think the idea of putting audit first in some cases is the way we'd need to progress. "Wow, what a blasphemy!", some might say ...  After all, if you have not defined controls, what are you going to audit? But remember that audit is meant broadly in this context and thus the opposite question is very relevant: what are you going to control if you have no idea what is going on? People sometimes define a security policy based on how things should be (and then WSJ happens - refresh your memory of the "WSJ saga" here), but then spent years trying to bring policy and reality together (and end up with an environment which is "half-controlled") Won't it be better to audit first, then control?  Or, at the very least, to run BOTH project at the same time so that improvements in control are audited and audit discoveries lead to control improvements!

Obviously, "do IDS before IPS" falls under the same principle: monitor first, [try to] control second. Here is another example: implement log management before identity management. Looking at logs will tell you what privileges the users actually use for doing their daily jobs. Then you can mix it up with what the idea access policy will be.

So, think about it! Questioning the common wisdom does often bring interesting insights.

Posted by on March 06, 2008 in Innovation , Log Management & Intelligence , Security | Permalink | Comments (0)

Anton Logging Tip of the Day #13: Into the Darkness ... or The Ominous World of Unix Binary Audit Logs

Following the new "tradition" of posting a security tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative.

So, Anton Security Tip of the Day #13: Into the Darkness ... or The Ominous World of Unix Binary Audit Logs

In this tip, we will take a peek at one of the most esoteric areas of logging: Unix binary audit logs. Solaris BSM and Trusted Solaris auditing is the least unknown :-) example of it, even though other Unix vendors have similar auditing capabilities - see this for HP-UX Audit and this for IBM AIX audit. Linux kernel audit is also pretty much the same thing. If you look for information on 'Solaris BSM audit logs' , you'd find plenty of tips on how to enable such logging, a little on how to manage/rotate the log files, a bit on how to survive the resulting data deluge and ALMOST NOTHING on what to do with the log data, which is kinda sad :-) After looking at BSM logs for a while, I developed an opinion that nobody has ever looked at them on a regular basis :-)

So, let's assume you enabled Solaris BSM kernel audit for user "root" and few other "interesting" users (there is no per-object logging in Solaris; other Unix variants do have it) via the following commonly recommended per-user configuration in /etc/security/audit_user:

root:lo,ad,fw:no

anton:all,-all:no

jsmith:all,-all:no

This configuration file pretty much leads to logging of everything that the above listed users do. Now, you have audit files growing like mushrooms in your /var/audit. What good does it give us?  First, we need to convert the binary audit files into text - something along the lines of

# auditreduce -A  /var/audit/20071127193515.not_terminated.SunUltra10 | praudit -l > /tmp/sol_box_11272007

will do. Now what? In this tip we will pick one thing to use these logs for: how use them to see who is trying to copy sensitive files off the system.

First, who is connecting out - lets's search the logs for 'connect' calls (if you are using LogLogic for it, use Index Search for this task; if not, grep will have to do, but be prepared to wait, possibly a looooooooooooong time :-)). A few recommended searches:

• "connect AND 172.16.10.*"   or "connect AND NOT 172.16.10.*"   (to look for connection to specific IPs or only to outside networks) or simply 'connect AND username' to look for connections by a specific user (if you log many users)

Here is an example found (with connect, IP and user in bold):

header,103,2,connect(2),,Tue Nov 27 11:36:46 PST 2007, + 193 msec,argument,1,0x4,so,socket,0x0002,0x0002,0x80d6,SunUltra10,0x0016,10.1.1.41,subject,root,anton,other,anton,other,29902,29720,0 1611 172.16.0.173,return,success,0

At this point we already know the user name of the user who run that connecting process since it will be in the results (you can also the user to search as I showed above).

Next, what are those connections - let's try to uncover which programs actually connected (BSM logs don't make that easy!). Let's search for process starts in the same time frame (within a few seconds):

• "execve AND NOT ls AND NOT <whatever other commands you don't care to see>" will give you a list of started programs.

Example found:

header,124,2,execve(2),,Tue Nov 27 11:36:46 PST 2007, + 115 msec,path,/usr/bin/scp,attribute,100555,root,bin,136,1573,0,subject,root,anton,other,anton,other,29901,29720,0 1611 172.16.0.173,return,success,0

OK, so somebody is trying to connect via SSH/SCP. Notice that both records - connect and execve -  have the same timestamp, up to a second. Sadly, time and parent process ID ( which is in our case 29720) is all that correlates them together.

Finally, what file was affected (i.e. copied off the system via scp in this case) - more digging is in order; we again use the process ID and time. The easiest is to search for a file name or browse all records around the same time frame (might be A LOT!):

• "*secret.zip* AND anton" will work; we can add the above process ID and look for "anton AND 29720" (but expect a lot of data since this is a shell process ID)

For example:

header,135,2,open(2) - read,,Tue Nov 27 11:36:47 PST 2007, + 900 msec,path,/tmp/not-so-secret.zip.gz,attribute,100600,anton,other,0,32743959,18446744073709551615,subject,root,anton,other,anton,other,29901,29720,0 1611 172.16.0.173,return,success,4

What do we know now? This user connected to this system and MAYBE copied this file via, MAYBE via scp. How cool is that? (A: not cool at all, since we are not sure, but that is all we can do with such logs)

To conclude, if you can avoid dealing with Solaris BSM logs, please do so :-) On a more serious note, you now know why these logs were called "the ugliest logs ever."  Even more seriously (but still pretty humorously), these logs are a classic example of trees that make every effort to obscure the forest, because these logs record system calls and not processes or user actions (and connect, execve and read are all logged separately). There are also many, many more idiosyncrasies where these come from :-)

Also, I am tagging all the tips on my del.icio.us feed. Here is the link: All Tips of the Day.

Posted by on November 30, 2007 in Compliance , Log Management & Intelligence , LogEd , Security | Permalink | Comments (0)

Protecting Logs from Admins: A Lost Battle?

One of the truly major challenges of log management is obtaining trusted logs of administrator activities, sometimes broadened to cover so-called "privileged users" (see my log trust pyramid post for more discussion). Many consider it to be the "lost battle" of logging. However, logging administrator access and actions is more important than ever today; it is one of the few workable ways of dealing with insider attacks.

So, I created this little table where I try to summarize some ways of protecting the C-I-A (confidentiality, integrity, availability) of various logs from administrators and privileged users (some successful and some very hard to pull off). Note that in the case of databases and applications, we need to protect their logs from DBAs and application admins and not from the underlying server platform admins.

UPDATE: table below might be corrupted in some blog readers; see the full table here.

UPDATE: table above might be corrupted in some blog readers; see the full table here.

Comments?

Posted by on November 26, 2007 in Innovation , Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

Simple Log-based User Profiling for Activity Monitoring

Recently I've been looking into detecting stolen/shared access account credentials. Now, how can you detect that? No NIDS will trigger, NIPS will let is pass, no unusual types of log records might ever by  produced (especially if only limited logging is enabled). And what raises the stakes is that this type of activity is not  only about "hacked" accounts, but also about insider abuse of accounts.

However, there will likely be changes in how normal log records are produced.

Let's summarize some known methods for using a simple user "profile" to detect account theft aka account sharing aka user impersonation aka access with stolen/shared credentials. It implies that we've been collecting the logs before the incident and have a solid trail of normal users and legitimate account owners.

1. Unusual login source IP (e.g. normal user always comes from 10.0.X.Y or 10.1.X.Y, but now we see a login from 172.16.0.Z) - will work in some cases, but not for the free-for-all servers such as at a University
2. Unusual login time (e.g. normal user always comes from 9AM to 5PM, now we see 3AM) - will work in most cases, but will fail if the attacker happens to be in the same time zone
3. Unusual login session length  (e.g. normal user always stays logged in for 5-20 minutes, now we see a 10 hour-long session) - works only if logout is logged; might not catch a lot of realistic but malicious connections
4. Unusual login frequency (e.g. legitimate user logs in once a day in the morning, now we see dozens of connections) - will work for some cases, but others will be missed
5. Unusual login failure/success ratio before a successful login (e.g. normal user always types the password right the first time, now we see failures than successes) - not too reliable, obviously
6. Unusual list of user actions performed (normal user only reads these files, but now we see writes to a very different set of files) - will work most frequently, but needs more granular logging of file access, object changes, etc (audit logging) [more on this in the near future!]

So, if you have logs of user activities, at the very least, logins and logouts ( but having records of more user activities is always better!),  for the last few weeks or months, one can compute the above profiles using historical data and then compare them with current numbers (very similar to some of the methods from my classic log mining presentation).

The final missing bit is for how long to collect your normal user behaviors: I discovered that 1 week to 1 month works pretty well. Less time yields unstable results and more time necessitates much more data crunching without much gain.

Posted by on October 12, 2007 in Innovation , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Log Trustworthiness Hierarchy

This post is inspired by the old TaoSecurity post Enterprise Trust Pyramid where he explains that he trusts some security, system and network evidence data more than other. For example, dedicated NSM sensor records are trusted more than vanilla server logs. With this post, I am looking to establish a log trustworthiness hierarchy so that people start thinking about trusting log data as a kind of a spectrum, from "probably trash" to "guaranteed to be an accurate record of activities."

So, do you trust your logs to accurately depict what happened on the system or network? Which logs do you trust the most? How do we increase this trust?

My first draft of such trust hierarchy follows below (from low trust to high trust):

1. Compromised system logs (mostly trash, but might contain bits that attacker missed/ignored)
2. Desktop / laptop OS and application logs (possibly changed by users, legitimate systems owners, etc)
3. All logs from others systems where 'root'/Administrator access is not controlled (e.g. test servers, etc)
4. Unix application logs  (file-based)
5. Local Windows application logs
6. Local Unix OS syslogs
7. Unix kernel audit logs, process accounting records
8. Local Windows server OS (a little harder to change)
9. Database logs (more trusted since DBA cannot touch them, while 'root' can)
10. Other security appliance logs (located on security appliances)
11. Various systems logs centralized to a syslog server
12. Network device and firewall logs (centralized to syslog server)
13. Logs centralized to a log management system via a real-time feed (obviously, transport encryption adds even more trust)

Admittedly, the differences between some of them are minor or even non-existent ...

To conclude, some logs DO in fact provide reliable evidence in case of an incident; you just need to know which ones to trust and which ones to only consider to be "hints" (or possibly even a misdirection). But of course, you need to first have logs and then look at them.

Posted by on September 27, 2007 in Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

More Web Proxy Log Tips

After publishing my proxy log tip (here), I figured I'd post a few more mini-tips on web proxy logging as well as a link to my full presentation (webcast with voice, slides only) on web proxy log management.

First, why look at proxy logs? Apart from my overall answer that applies to all logs, proxy-specific reasons are the following:

1. Review users’ activities on the web (not just surfing!)
2. Monitor applications' HTTP activity
3. Detect Web-enabled malware traffic
4. Study proxy performance metrics

Most people just focus on #1 above and kind of forget #2-#4. Also, the focus of #1 is often narrow - what do they surf at work?- and not broad - what do they do on the web? - which is much more useful. While there is no direct mention of proxy logs in recent regulations, monitoring what users do with YOUR data is clearly part of the compliance mandates (and, obviously, a good idea in general!) Indirect references to proxy logging can be seen in the following:

• Proxy monitoring is part of the overall control and governance (thus SOX, HIPAA, GLBA, etc)
• Legal requirements to have audit trails (thus HIPAA, PCI)
• Breach disclosure laws also impact (SB1386 and others, soon US-wide)
• On the flip side, privacy laws might mandate the opposite i.e. NOT having logs.

So, please treat proxy logs with the respect they deserve!  Here is my full presentation (webcast with voice, slides only), on analyzing and managing web proxy logs.

Related posts:

• Anton Logging Tip of the Day #12: Proxy Log Fun - Proxy Logs vs Information Leakage

Posted by on September 24, 2007 in Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Read "PCI Compliance" book chapter on logging!

As you know, the "PCI Compliance" book is out. Syngress/Elsevier publisher released one chapter from the book for free: and it is my chapter on log management and logging in PCI! Enjoy it here! [PDF] The book (and, in fact my logging chapter!) already got some glowing reviews.

While people still argue whether PCI is simple or overly complex, PCI DSS is certainly motivating a lot of people to take a long hard look at their IT security ... For example, more people are starting to look at database logs as a result of PCI. While some vendors are still lagging behind, you can get your database logs into LogLogic today!

Posted by on August 24, 2007 in Compliance , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Just What is "Scalability"?

Remember that discussion about syslog servers? While some people might try to sell you a mere syslog server for big money, others fall in the opposite end of the spectrum: they think that building a high-performance scalable log management system is as easy as installing a syslog on a machine ...

This fun story was related to me by Dimitri McKay, our network and systems engineer from the East Coast. He said: "Our biggest competitor in the field seems to be home-grown syslog servers. Although tons can be saved creating a generic "syslog server," at the end of the day, the resources and man-hours needed to maintain the thing offset the savings to total cost of ownership. I've seen several cases where a customer has a number of these servers running, 'grepping' their way through mountains of log data, and setting up script after script after script. Usually such a thing is started by someone very clever on that team, who gets bored, and moves on to something else, or some other job, and the syslog servers fall into disarray because nobody knows how and what to do with them.

When comparing an open source syslog server to a high-performance (75,000 log messages/second!) Loglogic ST, you do get what you pay for.

Specifically, one of our government customers hired a new employee who looked at the Loglogic ST3010 and expressed his disdain for commercial "out of the box" solutions. His exact words? "I can totally build that for like nothin'." So off our new friend goes, putting together a home-grown syslog server. 

First, he starts with a Linux kernel, installs a syslog collector, and then opens the flood gates direct from the Loglogic ST3010 via log message routing. The syslog server suddenly becomes unreachable, non-responsive to pings, SSH attempts, the whole nine. The flood gate of log traffic was then turned off.  Back to the drawing board.

Our new employee then comes back with two machines taped together in a cluster. The flood gates are opened, forwarding messages direct from the ST3010 to the home-grown solution, and the same result ensues. No response from pings, SSH, or even console access.

This continues on... three servers clustered, four servers clustered, five servers clustered... till the sixth server was added, and that's when our customer pulled the plug on his new hire. Too much time and hardware resources were being used in a project that was being done quickly and efficiently by one Loglogic ST3010 appliance. Why try to fix it, if it “ain’t broken!”

At the end of the day, you get what you pay for, and you pay for what you get."

Posted by on August 10, 2007 in Innovation , Log Management & Intelligence , LogMatters , Security | Permalink | Comments (0)

Anton Logging Tip of the Day #12: Proxy Log Fun - Proxy Log Analysis for Possible Information Leakage Detection

Following the new tradition of posting a tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative. One of the bloggers called it "pay it forward" to the community.

So, Anton Logging Tip of the Day #12: Proxy Log Fun - Proxy Log Analysis for Possible Information Leakage Detection

You probably know that web proxies (such as Squid, BlueCoat SG and others) produce a lot of detailed logs, that record all web traffic flowing through the proxy as well as pass/block decisions made by the proxy's content filters and possibly embedded anti-malware tools. Proxy logs can be used for a whole range of things, from routine monitoring for Acceptable Use Policy (AUP) compliance to malware detection as well as possibly looking for security scourge of 2007 - web browser attacks by malicious or compromised web servers.

Specifically, in this tip we will learn how proxy logs can be used for detection of file uploads and other outbound information transfers vie the web. First, think what is the legitimate use of file upload functionality in your environment. For example, if using web-based mail services is allowed, then sending an attachment will include an upload. What else? The rest will be considered at least suspicious...

In addition to file uploads, some malicious or commonly unauthorized applications will use similar methods to steal or transfer data, that will be reflected in proxy logs. Looking for HTTP methods (such as POST) and content-type in combination with either known suspicious URL  or user-agent (i.e. web client type) can often reveal spyware infections, actively collecting data. Admittedly, a well-written spyware can certainly fake the user-agent field so it is clearly not reliable, but still useful to add to our query above. 

So, here are some of the criteria we will use to look for information uploads in Squid and BlueCoat SG proxy logs:

• HTTP method (logged as "cs-method" by BlueCoat) = POST  (as opposed to the usual GET, used to retrieve web content).
• For information uploads: content type (logged as "RS(content-type)" by BlueCoat) = pretty much anything but "html/text" (which is the type used for uploading  web form contents) - especially try content types  "application/octet-stream", "application/msword", "application/powerpoint", "application/vnd.ms-excel", "application/pdf" and a few others to look for common file uploads.
• For spyware and application data transfers: user-agent set to anything but the common ones (i.e. not Mozilla, iTunes, LiveUpdate, etc) or even to "unknown." One can also try user-agent containing your favorite messaging app (e.g. "MSN Messenger", etc)  and see which such applications are in use.

(if you feel adventurous, other interesting content-types to try are "application/x-javascript" and "text/javascript")

Here are the examples found in proxy logs using the above query, including some "classics" (while spyware specimen are a bit dated, this method of  detecting them via logs is still relevant and useful):

1. 1124376766.026 RELEASE -1 FFFFFFFF 4734C557F9315105CA6BE0FA56B94D55 200 1124276674 -1 -1 unknown -1/0 POST http://reports.hotbar.com/reports/hotbar/4.0/HbRpt.dll
2. 1124392388.975 RELEASE -1 FFFFFFFF 810FFBF233584C330353CF0A8C31F5D2 503 -1 -1 -1 unknown -1/813 POST http://log.cc.cometsystems.com/dss/cc.2_0_0.report_u
3. 2007-05-19 03:55:12 160 10.1.1.3 - - - OBSERVED "Spyware/Malware Sources;Spyware Effects;Web Advertisements" - 200 TCP_NC_MISS POST text/html;%20charset=utf-8 http bis.180solutions.com 80 /versionconfig.aspx ?did=5342&ver=1.0 aspx - 10.1.1.2 273 175 - - none - -
4. 2007-05-21 03:10:40 4 10.1.1.3 Joanna- authentication_redirect_to_virtual_host PROXIED "Search Engines/Portals" - 307 TCP_AUTH_REDIRECT POST - http storage.msn.com 80 /storageservice/schematizedstore.asmx - asmx "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; MSN Messenger 7.5.0324)" 10.1.1.2 791 2566 - - none - -
5. 2007-05-22 21:35:09 215 10.1.2.237 200 TCP_NC_MISS 217 8122 POST http kenobi.example.com /exchange/john.smith/Drafts1/RE:%2520CustomerList.xls-2.EML - - DIRECT kenobi.example.com - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" PROXIED none - 10.1.170.42 SG-HTTP-Service - none –

The first three are traces of spyware (one was even identified by a BlueCoat content filter as "Spyware/Malware", the fourth is MSN Messenger-based activity while the fifth is emailing the Excel file via web mail.

Here are some other signs that will make the above log entry extra-suspicious is:

• A dead giveaway: upload happens to a "known bad" URL (e.g containing "gator" and others above) 
• Upload happens to an unresolved IP address (do a "whois" on it!)
• Uploads happens to a port not equal to 80 (i.e. the URL contains a port such as http://10.1.10.10:31337)
• Upload has confidential file name in the log entry (e.g. somebody dumb emailing a sensitive file to himself - as discussed here)

Overall, this log analysis method is good for casting a broad net to catch not just spyware-infected systems, but also unauthorized applications (e.g. method=POST and user-agent=iTunes), instant messaging (e.g. method=POST  and then by user-agent, content or URL), simple forms of data theft and document handling policy violations (emailing files to self via web mail: method=POST and sensitive file name present in the entry; also content-type set to popular Office file types) as well as other abuses of web access. As a result, proxy logs provide an extremely rich AND readily available source of data about threats that users face!

To top it off, one promising direction of future research is using web proxy logs to detect client-side exploits by malicious web servers (more on this in the near future!)

Possibly related posts:

• Anton Security Tip of the Day #6: The Other Web Log
• All my past security and logging tips

Also, I am tagging all the tips on my del.icio.us feed. Here is the link: All Tips of the Day.

Posted by on August 07, 2007 in Innovation , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Musings on 100% Log Collection

One of the most exciting, complicated and, at the same time, very common questions from the field of log management is the "what logs to collect?" question. This comes up during compliance-driven log management projects (in the form of "what to collect for PCI DSS compliance?") as well as operationally-driven (in the form of "what logs from this application do I need to detect faults and errors?") or security-driven log management projects (in the form of "which logs will help me during the incident response?")

What are the answers that one sometimes hear? Otherwise awesome log management guidance NIST 800-92 "Guide to Computer Security Log Management" confuses the reader with this fascinating blurb: "generally, organizations should only require logging and analyzing the data that is of greatest importance." And how do people to know which logs are of importance? (I did have a bit of a debate with NIST folks on that...)

Other answers are situation-specific and thus limited in their usefulness ("need IDS alerts + server logs to detect intrusions via correlation", "need all logs that show access to PHI").  I spoke about the pitfalls of "prioritizing before collection" in my presentation "Six Mistakes of Log Management" and its companion paper.  In some cases, such as the incident response scenario, you might be naturally leaning towards grabbing as much as possible, since you never know which bit will help you answer that dreaded "WHAT happened?!" question ...

On the other hand, there is a simple answer that doesn't suffer from the above issues: collect everything. However, many folks go into a state of shock upon hearing it :-) "Everything!?! HOW can you collect 'everything''? What about storage, bandwidth, hardware, etc?"

But you know what? It really isn't as bad as you think! Just think that:

1. Logs compress really well (1:10 to 1:15 compression ratios are not unheard of), so bandwidth and storage are less of an issue that initially estimated
2. Disk storage is cheap (and tape is cheaper still); holding a billion of log records might well cost  under $200 (!) in disk drive cost
3. Figuring out what you need, might need, can be "told to need", will need, etc is genuinely hard. You will never get it right!
4. Many logs don't need to get collected the next second after generation, thus allowing you to save some bandwidth by moving them when network is less busy.
5. Technologies exist to make sense of "everything", not just "hand-picked" and parsed logs. More on the "making sense" part later

Convinced yet? So, if you are pondering "what logs to collect?", try to switch your mindset into thinking "what will it take for me to collect everything?" You probably won't regret this decision!  At the same time, one can try to reverse this question and ask "why collect everything?" - in this case, the answer will be "because any other collection strategy is worse."

Related posts:

• Top 11 Reasons to Collect and Preserve Computer Logs

• Six Mistakes of Log Management

Posted by on July 19, 2007 in Compliance , Log Management & Intelligence , LogEd , Security | Permalink | Comments (0)

Top 11 Reasons to Look at Your Logs

As promised, I am following my Top 11 Reasons to Collect and Preserve Computer Logs with just as humorous and hopefully no less insightful "Top 11 Reasons to Look at Your Logs." 

1. The first reason is again disarmingly simple (is it, really? :-)). Read PCI DSS lately? Glanced at HIPAA? Suffer under FISMA? Yes, all of the above regulations say that you must not only have, but also review logs periodically.
2. Are your servers compromised now? How do you know if all your logs are stashed on a tape in a closet? Look at them! Now!
3. An incident happens. Really, who needs extra motivation to look at logs in such case? Using logs for incident response is a true "no-brainer" (however, you need to be pretty "brainy" to actually analyze them in case of an incident)
4. Users - from a CEO to a janitor. You do have to know what they do on your IT systems! How? Read the logs! Everybody leaves tracks.
5. Systems log plenty of errors. Sometimes they are silly, sometimes - benign. However, often they mean that "stuff" is about to hit the fan. Periodic review of logs reveals them and saves the day.
6. Network slowed to a crawl?  Applications are slooow? Server is not ... well, serving? :-) Where is the answer? In the logs, but you need to read them and understand them.
7. That policy you wrote a few months ago. Anybody following that? Anybody remembers that? Halloooo! Check the logs and you'd know.
8. By now you know that your auditor might ask for your logs. But do you know they might also check whether you looked at them? Do you? Review the logs and leave the record of this activity in the logs.
9. Change can be good. But then again, it may be the sign that your controls are lacking. Who changes what and when? From what and to what? Just review the logs.
10. Now, you hate looking at logs. You have too many (as if everybody else doesn't...)! In this case, look at a specific subset of logs that you never saw before- NBS. Or just deploy log management that can do it for you.
11. Logs can help you predict the future (if you review, know and love them :-)). Don't believe it? If you read them for long enough, you develop an ability to predict the future, albeit mostly future problems :-)

See also: Top 11 Reasons to Collect and Preserve Computer Logs. 

Coming soon: "Top 11 Reasons to Secure and Protect Your Logs"!

Posted by on July 06, 2007 in Log Management & Intelligence , LogMatters , Security | Permalink | Comments (0)

Defend In Depth

Earlier in June, Mark Ford, who lead's Deloitte and Touche's Security and Privacy Services practice, was interviewed by IT Security.

According to Ford, IT security is one of the top issues for CIOs. However, he also noted that a corporate focus on IT security is quicker to take hold in information-centric organizations (banks and other financial service organizations, for example) than in industries like manufacturing or retail that are more geared towards consumers and whose focus is on business operations. With the increase in regulatory focus of such mandates as PCI-DSS and SOX, this has changed over the past few years as corporations in a variety of industries need to have strong IT security in order to be compliant.

What does this mean? Ford commented that historically, companies have put emphasis on the perimeter threat as the key component of IT security. But now, security emphasis is shifting towards a layered defense of the IT infrastructure. The perimeter is still important, but can no longer be considered the main component.

A shift away from the traditional security approaches and towards log management and intelligence, which allows for just the layered sort of approach Ford approves.

Technorati : Compliance, LMI, Log Management, PCI Compliance, Sarbanes-Oxley

Posted by on June 25, 2007 in Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

Webcast: Protecting Information Assets

Most enterprises are worried about protecting critical confidential information and customer data. How are you doing it? What are industry best practices for securing infomation assets?

Join us during a daylong event on securing your data at SCMagazine -- along with our partners NetApp and Websense -- on a webcast on June 20, 2007 from 10:30 am PST / 1:30 pm EST. The event will cover such log management topics as:

Aggregating and storing a "fingerprint" of all systems and user activity.
Learn best practices and mandates for log data retention.
Monitoring access to information stored in your enterprise.
Understand how to alert and report on the flow of information across multiple systems and platforms.
How does log data work together with information leakage solutions to prevent privacy violations?
How can you protect chain of custody and ensure that the information will stand up as evidence in the court of law?
How quickly should you be able to produce and share reports?
What are common reports and alerts being used for information asset and compliance enforcement?

Log data is the digital equivalent of a surveillance camera. It functions a deterrent and also provides legal evidence to prosecute those who leak or steal information. In this special Webcast, subject-matter experts from LogLogic, WebSense and NetApp will discuss how organizations can ensure IT Governance, Compliance and mitigate risk with multiple mandates using next generation Log Management and Intelligence, Data Leakage Prevention and high performance and high-security records retention.

Register here.

Technorati : Compliance, LMI, Log Management

Posted by on June 19, 2007 in Compliance , Risk Management , Security | Permalink | Comments (0)

California Considers Making PCI DSS Law

This week California is considering a bill that would require organizations that accept credit and debit cards to follow the Payment Card Industry (PCI) Data Security Standard. Noncompliance could mean banks would have to cover the costs associated with notifying customers that their credit card numbers may have been stolen and the cost of replacing credit cards, at a cost that could run upwards of $1 million per breach, according to estimates in a California State Senate report in May that provided details on the bill. The California law would apply to anyone who wanted to do business with a California resident, according to this article at Government Executive blog.

The public backlash after the January disclosure of a major security breach by Massachusetts based retailer TJX has acted as a stimulus for attention and consumer protection mandates. Just last week Minnesota enacted the Plastic Card Security Act, based on the PCI Standard. And other states like Massachusetts and Texas are also considering laws. The Lone Star state's House voted unianimously to approve the PCI-related bill, but the state Senate closed its session before it could vote on the issue.

Log management can help out with complying with the PCI DSS regulations quickly, plugging into your existing IT infrastructure. For some tips, check out 7 Habits of Highly Effective PCI Compliance- a Forrester Webcast with analyst Khalid Kark, sponsored by LogLogic. A PCI book is on the way from LogLogic's Anton Chuvakin later this year.

Technorati : Compliance, Log Management, PCI

Posted by on June 07, 2007 in Compliance , Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

Why Banks Are Moving to Log Management

Banking Information Security magazine is covering the emerging trend in log management as it makes it way through the banking sector. They profile LogLogic cusomer, Citizens & Northern Bank, a $1.2B bank out of Pennsylvania, that has made log management a requirement for meeting compliance mandates with Gramm-Leach-Bliley and Sarbanes-Oxley. Using log management, the bank's auditors now have a way to easily track and monitor log data and get compliant fast.

Citizen's Bank learned early on what other G2000 companies are now realizing -- log management is inceasingly becoming the weapon of choice in the quest for compliance. Banking, an industry known for security and scrutiny of IT products is joining a growing trend of deploying log management for security, forensics, loss prevention and compliance. Why? As the article explains, the Industry's Federal Financial Institutions Examination Council (FFIEC) says that "without real log management, organizations are out of compliance and at risk" and calls on companies to monitor their log data. From the article,

"As administrators responsible for various network devices and operating systems, we need to know what typical behavior is," says Pete Boergermann, head of MIS at Citizens & Northern. "When we look at events, we are more apt to know what we are looking at and respond."

Read more about Citizen and Northern Bank's log management deployment in this SANS What Works case study from last year. Log Management and Intelligence is on the IT and business agenda. Industry trends are available in the just-released market survey on log management adoption. The LogLogic-sponsored research study with the SANS Institute is available for preview here or join us with SANS for a presentation of the trend at a joint webcast.

The complete article is at Banking Information Security, please note that registration may be required.

Posted by on June 04, 2007 in Compliance , Log Management & Intelligence , Risk Management , Security | Permalink | Comments (0)

G2000 Logjam Continues To Spur Log Management Says SANS Survey

We teamed up with the SANS Institute again this year to survey the G2000 on the trends driving log management and intelligence. You can dowload a copy of the preliminary findings of the 2007 Log Management Survey or sign up to attend a webcast presentation of the results with SANS on June 6th.

Based on last year's findings SANS Institute CEO Stephen Northcutt said:

"In our analysis, we found that G2000 IT leaders are today not only validating the use case for log intelligence, but are signaling the increased need for the market to evolve even further with log services that cost-effectively allow them to exchange information with other solutions."

The 2007 survey saw that trend continue to evolve. Polling more than 650 IT professionals in government, financial services, banking, manufacturing, healthcare, telecommunications, and education sectors from the North American Global 2000 (G2000) - Forbes's comprehensive list of the world's biggest companies, key findings:

· Log data monitoring continues to grow exponentially. Of those surveyed over 61% report using log data to assess IT incidents and minimize downtime (an increase of 24% over 2006 survey results).

· Log data retention is up significantly, but most of the G200 and G2000 are still failing to meet compliance regulations. Despite regulatory recommendations or requirements that logs be maintained for three to five years, 11% say they keep log files between 30 and 90 days, 10% retain data for six months and 5% less than 30 days. Remarkably, a full 14% say are not sure how long they keep log data, relying on the system default as defined by their operating system.

· Security while important is not the prime motivation for log management. More than half of those surveyed reported operations management and monitoring the health of the network as the prime motivation for using log data. 43% indicated compliance with SOX, PCI and other mandates as the top priority.

· The quantity of stored log data is rising. 57% percent of survey respondents store logs from as many as 500 sources.

Who:     Dr. Anton Chuvakin, Director of Product Management LogLogic
         Mike Rothman, President and Principal Analyst, SecurityIncite,
         and author of The Pragmatic CSO

What:    Online webinar roundtable discussing log management trends and
         best practices.

When:    May 31, 2007, 2 PM ET / 11 AM PT

Registration: www.loglogic.com

Visit loglogic.com

Subscribe to this blog’s feed

• September 2010
• August 2010
• July 2010
• June 2010
• May 2010
• April 2010
• March 2010
• February 2010
• January 2010
• December 2009
• November 2009
• October 2009
• September 2009
• August 2009
• July 2009
• June 2009
• May 2009
• April 2009
• March 2009
• February 2009
• January 2009
• December 2008
• November 2008
• September 2008
• August 2008
• July 2008
• June 2008
• May 2008
• April 2008
• March 2008
• February 2008
• January 2008
• December 2007
• November 2007
• October 2007
• September 2007
• August 2007
• July 2007
• June 2007
• May 2007
• April 2007
• March 2007
• February 2007
• January 2007
• December 2006
• November 2006
• October 2006
• September 2006
• August 2006
• July 2006
• June 2006
• May 2006
• April 2006
• March 2006
• February 2006
• January 2006
• December 2005
• November 2005
• October 2005
• September 2005

Blogroll

• Whatever Compliance
• CynergisTek
• Bruce Schneier
• InfoSecDaily

Compliance

• Compliance Pipeline
• SOX Compliance Journal
• Sarbanes-Oxley 101

Good Reading

• Internet and Web Security Review
• Log Management ROI
• Log Management Best Practices
• SANS Institute Log Management White Paper

LogLogic

• O’Reilly
• Anton Chuvakin
• LogLogic.Com
• Jian Zhen

LogLogic Partners

• Juniper Networks
• ONStor
• Counterpane
• Bluecoat

Sites We Watch

• Verizon Business Security Blog
• SANS
• IT Manager's Journal
• CIO Insight
• Security Focus
• TechWeb
• Slashdot
• InformationWeek
• The Reg
• C/Net
• OSF DataLossDB | Data Loss News, Statistics, and Research