Do Lambs Grow On Trees?

By Guy Churchward, LogLogic CEO.

According to the Foods and Nutrition Encyclopedia, the earliest cultivation of cotton discovered in the Americas occurred in Mexico, some 8,000 years ago.

Cotton was first cultivated in the old world 7,000 years ago by a civilization comprised of parts of eastern Pakistan and northwestern India.

During the late medieval period, cotton became known as an imported fiber in northern Europe, without any knowledge of how it was derived.

John Mandeville, wrote in 1350, "There grew in India a wonderful tree which bore tiny lambs on the ends of its branches. These branches were so pliable that they bent down to allow the lambs to feed when they are hungry.

Denim is a rugged cotton twill textile, in which the weft passes under two or more warp threads. This produces the familiar diagonal ribbing identifiable on the reverse of the fabric.

The word comes from the name of a sturdy fabric called serge, originally made in Nîmes, France, by the Andre family. Originally called 'serge de Nîmes', the name was soon shortened to denim.

The contemporary use of “jeans” comes from the French word for Genoa, Italy (Gênes), where the first denim trousers were made.

A similarly woven traditional American cotton textile is the diagonal warp-striped hickory cloth that was once associated with railroad men's overalls. Hickory cloth was characterized as being as rugged as hickory wood.

The staying power of jeans started me thinking why despite consumers transient disposition they've thrived thus my thirst for their inception and journey.

So is it durability, applicability and flexibility that drives the Jeans phenomena and pervasive utilization in a myriad of applications from work to high fashion.

At its core, I believe it's solid, reliable and pervasive, the manufactured resource is seemingly endless, and the applications are only limited by our needs and imagination.

Jeans have gone through many styles driven by need such as flares for sailors who work in the boiler rooms and need ventilation to the skinny jean tailoring to quench the thirst of fashion-forward differentiation.

Manufacturers like Wrangler, Lee and Levis prevailed, albeit in for the practical and budget conscious.

7-For-All-Mankind, Rock-N-Republic and True Religion captured the hearts and wallets of the discerning buyer.

The trend-pariahs provide fad driven designs and mass market retailers muscle in on the action with lackluster products barely cloaked by an impressive array of billboards and paid applause.

Both fail spectacularly in delivering the true quality and lasting style yet capture ink and sweep up unsuspecting spectators in a fervor of shallow promises.

When the dust settles you are left with an uncomfortable, badly fitting pair of jeans with excess embellishments calling you screechingly from the darkest reaches of your wardrobe.

Our market-space is very much like this, IT assets are cotton plant, pervasive and an unlimited source of raw materials.

Logs are like denim, strong, core and foundational. Jeans are an artifact, the way you use the materials, like compliance, forensics, and security event management.

Like the adoption of Denim, there are pockets of usage and now the generalization of this true value of denim is becoming apparent. The rail roads found PCI, the French found event management and the Indians forensics.

LogLogic  has stayed true to the value of denim and have been working the mills for over 7 years.

One competitor has spent years discrediting the value in Jeans, they've been making shirts and doing well at it. It does, however, seem that their shirt sales are slowing so they've conveniently forgotten they deplore Jeans, rushed out an offering, put their billboards out and are now claiming leadership.

Another seems oblivious of the structure or durability needed, instead they focus on the sparkle of their embellishments. I was told recently by a shared client that they were initially euphoric at the promise but have since suspended further use. Another told me they get free T-Shirts instead of fixes, half the company have been augmented but still no working product.

Everyone has denim in their wardrobe, the question is, have you really seen how valuable a well cut well made pair of jeans can make you look?

Cheap is a short term win in a long term market and a branded, badly constructed pair will give your backside the appearance it's the size of a small elephant… not to mention the possibility of public embarrassment if the coverage isn't quite what it seems..

John Mandeville was misguided. Let's make sure the euphoria, fervor and promises are measured and fully understood.

We pioneered and lead this market. We are the first and only company who specialize in log and compliance management for the enterprise.

We are your favorite pair of utilitarian jeans that form fit, flatter and serve as casual yet elegant attire. We've stayed true to denim. We're your form fit comfort jeans that are casual yet smart.  But we're not arrogant, we know change comes to everything and we intended to continue to lead and not follow. This week's 3.4 release is the first of a new wave of thinking for this industry. We driving innovation so you always look properly dressed whether laying pipes or attending the opera.

Posted by Bill Roth on April 30, 2010 in Guy , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Logging Poll #8 - Log Security and Protections

My next logging poll is out - with it I set out to figure out the old mystery of mine, why people don't protect their log data (e.g. see this lamentation "Top 11 Reasons to Secure and Protect Your Logs")

Vote away! As always, results will be posted.

Past polls and analysis are all here.

Posted by on August 05, 2008 in Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Cross-Device-Type Log Management vs Device-Specific Log Management

Now, I have to first admit that, in general, dealing with logs on a device-specific basis is a utterly inefficient. What I mean here is when you gather Windows logs in one place, Linux logs in another place, database logs in yet another place; all in different formats, all in different systems not connected to each others, all managed by different people who don't talk to each other (and sometimes hate each other). Basically, this situation is "logs at their worst": all different, all disjointed and, as a result, all next to useless.

However, there are rare situations where you can choose device-specific log management approach without wasting time and money. For example, you might be motivated by the fact that tools that can handle one specific type of log data (e.g. Windows-only, web server-only or Cisco PIX-only) are usually many times less expensive than cross-device log management tools. The table below clarifies it:

Use Case vs Approach	No log consolidation - logs remain on systems they are produced	Device-specific log consolidation and analysis	Cross-device log consolidation and analysis from all log sources
Alerting based on log strings (keywords) that indicate security or operational problems	Impossible or tremendously hard to manage across many systems	Acceptable - alerts on each log type are handled by different teams	Superior - all logs are available for analysis when the alert is triggered
Reviewing logs for troubleshooting server problems	Acceptable - server logs are sufficient for	Better - one can also look at logs from other similar servers	Better - but same as device-specific log analysis since only one type of logs needs to be reviewed
Log review for compliance with PCI DSS	Not acceptable - log management is mandated by Req 10	Impossible or very inefficient - as many types of log data needs to be collected and reviewed	Optimal - all PCI relevant logs can be collected and reviewed in one system
Looking for records of a specific user activity	Impossible or tremendously hard since hundreds of systems might need to be searched	Inefficient - several different systems needs to be accessed to review all records of user's activities (and then data needs to be manually correlated)	Optimal - one query gives all traces of the user activity
Log review for incident response or forensics investigation	Impossible or tremendously hard since hundreds of systems might need to be searched for evidence	Inefficient - several different systems needs to be searches for evidence and then data manually correlated	Optimal - all log evidence can be found, reviewed and analyzed on one system, neither hundreds, nor several

Also, while looking at logging tools, one needs to make a distinction between tools that can collect all sorts of logs but only allow you to analyze one log type at a time (e.g. sawmill) vs tools that can collect all sorts of logs AND allow you to analyze all of them together (e.g. LogLogic). The former tools still fall under "device-specific log management" despite their ability to gather hundreds of different log types.

The bottom line: in most cases, cross-device, uniform log management provides huge value, especially if your motivation for log management is compliance or incident response.

Posted by on June 02, 2008 in Innovation , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Quick and Fun Poll on Behavior Change

While I am analyzing my previous poll, here is a quick and fun one: do we change our behavior when monitored? Vote away!

 

Posted by on March 27, 2008 in Compliance , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Top 11 Reasons to Analyze Your Logs

As promised, here is another "Top 11 Reasons" which is about log analysis. Don't just read your logs (definitely don't just collect them); analyze them. Why? Here are the reasons:

1. Seen an obscure log message lately? Me too - in fact, everybody have. How do you know what it means (and logs usually do mean something) without analysis? At the very least, you might need to bring additional context to know what some logs mean (example: IP address -> hostname -> server owner)
2. Logs often measure in gigabytes and soon will in terabytes; log volume grows all the time - it definitely passed the  limit of what a human can read a long time ago, it then made simple filtering 'what logs to read' impossible as well: automated log analysis is the only choice.
3. Do you peruse your logs in real time? This is simply absurd! However, automated real-time analysis is entirely possible (and some logs do crave for your attention ASAP - e.g. major system failures, confirmed intrusions, etc)
4. Can you read multiple logs at the same time? Yes, kind of, if you print them out on multiple pages to correlate (yes, I've seen this done :-)). Is this efficient? God, no! Correlation across logs of different types is one of the most useful approaches to log analysis.
5. A lot of insight hides in "sparse" logs, logs where a single record barely matters, but a large aggregate does (e.g. from one "connection allowed" firewall log to a scan pattern). Thus, the only way to extract that insight from a pool of data is through  algorithms that "condense" that collection of logs into usable knowledge (some say, visualization is the way to go)
6. Ever did a manual log baselining? This is where you read the logs for a while and learn which ones are normal for your environment. Wonna do it again? Thought so :-)  Log baseline learning is a useful and simple log analysis technique, but humans can only do it for so much before burning out.
7. OK, let's pick the important logs to review. Which ones are those? The right answer is "we don't know, until we see them." Thus, to even figure out which logs to read, you need automated analysis.
8. Log analysis for compliance? Why, yes! Compliance is NOT only about log storage (e.g. see PCI DSS). How to highlight compliance-relevant messages? How to see which messages will lead to a violation? How do you satisfy those "daily log review" requirements (again, see PCI DSS)? Through automated analysis, of course!
9. Logs  allow you to profile your users, your data and your resources/assets. Really? Yes, really: such profiling can then tell you if those users behave in an unusual manner (in fact, the oldest log analysis systems worked like that). Such techniques may help reach the holy grail of log analysis: have the system automatically tell you what matters for you!
10. Ever tried to hire a log analysis expert? Those are few and far between. What if your junior analysts can suddenly analyze logs just as well? One log analysis system creator told me that his log data mining system enabled exactly that. Thus, saving a lot of money to his organization.
11. Finally, can you predict future with your logs? I hope so! Research on predictive analytics is ongoing, but you can only do it with automated analysis tools, not with just your head alone (no matter how big :-)) ...

 Past top 11 reasons:

• Top 11 Reasons to Collect and Preserve Computer Logs
• Top 11 Reasons to Look at Your Logs 
• Top 11 Reasons to Secure and Protect Your Logs

 

Posted by on February 22, 2008 in Innovation , LogMatters | Permalink | Comments (0)

Poll: What logs do you actually LOOK at?

This is my 6th logging poll (vote here now!)- links to the previous five polls below.

This one is deceptively similar to the #1 below, but it is not. This poll is What logs do you actually LOOK at? and not Which Logs Do You Collect? In other words, are you a log packrat? Are you collecting and never using the log data? You are making a mistake, if you don't.

Past polls:

Poll #5 "What are your top challenges with logs?" (analysis)

Poll #4 "Who looks at logs in your organization?" (analysis)

Poll #3 "What do you do with Logs?" (analysis)

Poll #2 "Why collect logs?" (analysis)

Poll #1 "Which logs do you collect?" (analysis)

 

Posted by on February 14, 2008 in Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Logging Poll #5 "Top Logging Challenges" Analysis

OK, this poll was insightful! The raw results are here and below:

What can we learn from this?

First, what are the top challenges? It is with great regret :-) that I report that the #1 challenge is exactly the one I thought it would be: We collect logs but don't have time/resources to look at them. Yes, automated "analysis challenge" has only become more of a challenge as people deploy more tools that enable log collection on a massive scale (e.g. 75,000 logs/second). I dare to predict that we will finally have to tackle this one in the next year or two. In fact, this challenge rears it ugly head via another popular response, Lack of log analysis tools, which made Top 5 responses as well.

Second, even though I didn't make any predictions about the #2 entry, but I was surprised: No way to effectively search all logs is a  very close #2 (obviously, 1 vote is not statistically significant here). Indeed, log searching is an elusive little problem, especially when we want to do it fast and on a large pool of logs. Even though I think we need to search less and discover more, the need to search logs will be with us forever (and, no, I don't think you need a special product just to do that ...)

Third, I am happy to report that this poll indicates that we finally broke the back of "the beast" of  not having logs. Responses that point at not having logs (e.g. Logging is not enabled, We don't know what logging we must enable,  etc) are not terribly popular (then again, maybe it is due to self-selection of the blog readers ...)

Fourth, infrastructure! Specifically, No infrastructure to manage the log volume we have is very popular as well (#4). This proves the point that I used to not take very seriously in the past (by mistake): when megabytes become gigabytes and those grow into terabytes, many things that used to trivial (e.g. moving logs from A to B, saving logs to disk, etc) become grand engineering challenges... Indeed, to manage high volume of logs you need a scalable log management solution (example)

Sixth, as I lamented, few care about log security (this counts as laments, I guess).  Secure storage of logs is only bothering a few people. One word: yet! As of today, stored log hashing + (sometimes!) log transport encryption + (rarely!) encrypted archives are the state of the art.

Next poll is coming up!

Posted by on February 08, 2008 in Innovation , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

NERC CIP Rules Out - Logs In!

NERC security rules [PDF], that were updated and became mandatory last week, might well become "a new PCI DSS" and trigger "a golden age" of security in the energy industry: the rules are mandatory, they are specific (more specific than a lot of other regulatory security guidance) and there is an enforcement body (NERC) that can make life miserable for those not complying. 

Here are some log-related examples from the guidance:

"R5.1.2. The Responsible Entity shall establish methods, processes, and procedures that generate logs of sufficient detail to create historical audit trails of individual user account access activity for a minimum of ninety days. "

"R6.4. The Responsible Entity shall retain all logs specified in Requirement R6 for ninety calendar days."

"R6.5. The Responsible Entity shall review logs of system events related to cyber security
and maintain records documenting review of logs. "

So, again: have logs, retain them ("Top 11 Reasons to Collect and Preserve Computer Logs") and review them ("Top 11 Reasons to Look at Your Logs"). Or, better, have a log management tool do it for you!

 

Posted by on January 24, 2008 in Compliance , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Poll: Who looks at logs at your organization?

Here is my next poll about logs: Who looks at logs at your organization? Vote here! Also, my past polls and analysis are here.

Past polls were:

Poll#3 "What Do You Do With Logs?" (analysis)

Poll#2 "Why Collect Logs?" (vote here, results so far, my analysis)

Poll #1 "Which Logs Do You Collect?" (vote here, raw results here, analysis here)

Posted by on December 19, 2007 in Log Management & Intelligence , LogMatters | Permalink | Comments (0)

On Approaches to Database Monitoring

So, people sometimes ask me about how to do database logging/auditing/monitoring and log analysis right. The key choice many seem to struggle with for database auditing and monitoring is reviewing database logs vs sniffing SQL traffic off the wire.  Before proceeding, please look for more background on database log management, auditing and monitoring in my database log management papers (longer, more detailed - shorter)  The table below summarizes the situation with database monitoring and auditing - now you can make your choice more intelligently (items in bold are the ones I consider key):

 

	Pro	Con
Sniff SQL traffic from the wire	No database performance impact Awareness of returned content (for SELECTs) Guaranteed role separation Better for DBA monitoring No agents No database configuration changes	Extra device needs to be purchased, deployed and managed Doesn't work with encryption No local access monitoring
Collect and analyze database logs	No extra $$$ - use your existing logging tool Can user review activity across log sources, from databases to servers Satisfies compliance demand for "database log review" Can monitor ALL access to data in the database, even over APIs and local	Performance impact possible (*) Database config changes needed Usually not truly "real-time" (polling)

Choose logs if you care for the relevant Pros (esp key ones) associated with them; choose sniffing if you care for the Pros and are NOT undermined by their Cons (e.g. difficulties of supporting encrypted traffic)

Of course, one can also opt for a combined approach which follows the ideas of "double the benefits - for double the cost"...

(*) Nobody really knows what it will be in each particular situation: 0-40% were observed under various conditions by various people ...

Posted by on December 17, 2007 in Innovation , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Again, On Criticality of Logs

I just wanted to highlight two pieces that, again, speak - or, better, scream! - about the importance of logs. I suspect that LogBlog readers don't need additional motivation to take logs seriously, but these are just too useful to skip.

First is the interview with some convicted criminal hacker, who said that '... it would have been easy for IT and security managers to detect him in their companies' systems ... if they'd been looking. The problem was that, generally, no one was paying attention.

"If they were just monitoring their boxes and keeping logs, they could easily have seen us logged in there," he said, adding that IT could have run its own scans, checking to see logged-in users."'

Amen to that! Indeed, many of the successful-and-then-undetected attacks are due to incompetence. Why? 'Cause lacking logs or ignoring logs is indeed negligent and incompetent!

Second, is my comment on the TJX case, which kinda follows the same idea: 'Dr. Anton Chuvakin, a security expert with LogLogic, said TJX [probably] didn't have decent logs. "What took TJX months was looking at all their systems and determining who took what data, from where, where it was sent, etc. The investigation took them months. They likely didn't have any logs, because they had to do system forensics rather than log analysis to arrive at their conclusions about who stole the data and how. If they had collected and analyzed log data centrally, the investigation would have been a piece of cake," he said in an e-mailed comment to InternetNews.com.'

Indeed, doing disk forensics to know who did what is much more painful than checking reliable logs. Save yourself by logging, then saving and reviewing the logs!

So, one more time (not the last, mind you!):

• Make sure you have and collect logs (here are 11 reasons for that)
• Make sure you look at logs (here are the 11 reasons)

Posted by on December 11, 2007 in Log Management & Intelligence , LogLogic News , LogMatters | Permalink | Comments (0)

Logging Poll #3 "What Do You Do With Logs?" Analysis

So, the results of my 3rd poll are ready: live results are here, picture is also in this post.

First, this poll way more popular than my previous "why" poll. It seems like people do dislike to wonder "why."

Second, what are  the two choices, that are by far the most popular? They are:

• Store raw logs on a server (23%)
• Search raw logs (grep) when needed (24%)

Yes, this is the "state of the art" of logging:   collection of raw logs and "as needed" grep aka "slow and painful" search. In fact, the above answers might not even be given by the same people: some might be grepping logs on the individual servers, while others collect them on syslog servers and never touch them. That is why being in log management business is such a great thing: you have nearly the whole world to evangelize about the value of logs and log management tools.

Third, what's the next most popular idea of analyzing logs? It is "Run my own log analysis tool" at 10% of the respondents. Indeed, this movement still lives and thrives: people choose the Build->Suffer approach to log management often enough ...

Fourth, next come my somewhat self-inflicted surprise: apart from commercial log management (at 4%) and rolling one's own (discussed above at 10%), I added the option of "Use other log analysis tools"   which captured 7% of the vote. But what does that mean? I have no idea!

Fifth, I am NOT surprised by the lack of popularity of the rule-based correlation tools, such as SIEM (at 2%). When I made my decision to join LogLogic, I had to ponder this one really, really hard. My conclusion at the time (which is also valid now, even more than back in 2006) was that "SIEM is for some, log management is for everybody." This poll confirms this further.

Finally, all my logging polls and analysis are here. Next one is coming up!

Posted by on December 08, 2007 in Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

Imagining an Ideal Log Management Tool?

The idea came from Jeremiah Grossman (here) when he described "The Best Web Application Vulnerability Scanner in the World" thus: "Within a few moments of pressing the scan button it’ll find every vulnerability, with zero false positives, generate a pretty looking report, and voila you’re compliant with GLBA, HIPAA, and PCI-DSS. Of course, we all know such a web application scanner is simply not possible to create for a variety of reasons."

So, let's imagine the ideal log management application.

1. Logging configuration: the ideal log app will go and find all possible log sources (systems, devices, applications, etc) and then enable the right kind of logging on them according to a high level policy given to it
2. Log collection: it will collect all the above logs securely (and without using any risky super-user access ) and with little to no impact to networks and systems
3. Log storage: it can security store the above logs in the original format for as long as needed and in a manner allowing quick access to them - in both raw and summarized/enriched form
4. Log analysis: this ideal application will be able to look at all kinds of logs, known to it and previously unseen, from standard and custom log sources, and tell the user what they need to know about their environment and based on their needs: what is broken? what is hacked? where? what is in violation of regulations/policies? what will break soon? who is doing this stuff? The analysis will power all of the following: automated actions, real-time notifications, long-term historical analysis as well as compliance relevance analysis
5. Information presentation: this tool will distill the above data, information and conclusions generated by the analytic components and present then in a manner consistent with the user's role: from operator to analyst to engineer to executive. Interactive visual and drillable text-based data presentation across all log sources. The users can also customize the data presentation based on their wishes and job needs, as well as information perception styles
6. Automation: the ideal log management tool will be able to take limited automated actions to resolve discovered and confirmed issues as well as generate guidance to users so that they know what actions to take, when full-auto mode is not appropriate. The responses will range from full-auto actions to assisted actions ('click here to fix it') to issuing detailed remediation guidance. The output will include a TODO-list of discovered items complete with actions suggested, ordered by priority
7. Compliance: this tool can also be used directly by auditors to validate or prove compliance with relevant regulations by using regulation-specific content and all the collected data. The tool will also point at gaps in data collection as it applies to specific regulations that the user is interested in complying

In other words, this magic black box will have raw log data shoveled from one side and will have answers to key questions that concern the IT users and their managers on the other side.

Am I nuts? Well, can I dream for a second? :-)

Posted by on November 06, 2007 in Innovation , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Top 11 Reasons to Secure and Protect Your Logs

Following my now-famous Top 11 Reasons to Collect and Preserve Computer Logs and Top 11 Reasons to Look at Your Logs, here is the promised "Top 11 Reasons to Secure and Protect Your Logs"

1. Let's review why you are reviewing logs. Will logs that might have been changed by somebody, somewhere, somehow still be useful for items 1-11 from here? No? Secure them!
2. Logs in court? Challenges abound! To respond to them, one needs to protect the logs so you can claim that they are both authentic and reliable.
3. A human error still beats an evil hacker as the main cause of IT problems. Are your logs safe from it? Available when needed? Protect them from crashes and other faults!
4. PCI DSS just says so: "Secure audit trails so they cannot be altered." Want to do it- or pay the fines?
5. Do you protect financial records? Identity info? Passwords? Some of it ends up in logs (even if by mistake) - thus making them sensitive, protected information. Secure the C-I-A of logs!
6. Do you look at logs during incident investigation? Do you want them to be "true" or full of random (if creative...) "stuff", inserted by the guilty party? Secure the logs!
7. Think that "attacks vs logging" are theoretical? Think again. Are your logs safe or vulnerable? Is your logging tool 0wned?
8. Syslog + UDP = log injection + log loss. Are you protected (reliable TCP, confirmed delivery, encryption - SSH, SSL, VPN)?
9. Why modify logs? No, really, why change logs? If you never change logs - and you never should, unless it is appending to them - hash them right away after collection to make them immutable.
10. Logs are backed up on tape - who will see them? Well, whoever restores the tape, that's who! Encrypt them to protect them from accidental and malicious disclosure if tape is lost.
11. Why log access to logs? Same reason why you had the logs in the first place - to review who did what. Who broke through and stole the logs? Who browsed them without permission? Only logs will tell - if you have them!

Overall, one need to strive for having no holes in log safeguards from log birth to analyst conclusion based on log information...

Possibly related posts:

• Top 11 Reasons to Look at Your Logs

• Top 11 Reasons to Collect and Preserve Computer Logs

Posted by on November 02, 2007 in Innovation , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Just What is Enterprise Class? Part V

Here is the final enlightening post from Dimitri McKay, our super-brilliant network and systems engineer from the East Coast, where he continues to discuss the meaning of the phrase "enterprise-class," which is certainly MUCH more than a marketing buzzword! Part I is here, Part II is here, Part III is here.  Part IV is here.

Dimitri McKay says: "This is the final piece in the five part series entitled “Enterprise Class”. The term “Enterprise Class” for most of us has appeared on Bullshit-Bingo cards for years, but when we actually explore WHAT “Enterprise Class” is and how it pertains to a certain product, you can learn quite a bit about that product, and how it stands up to both the competition and how it stacks up against what it is and what it should be.

As of this writing, LogLogic has set free it’s next code-base, LogLogic4 Release 2. With added features and fixes, LogLogic fits the Enterprise all the more with each release.

13. Usability: Because of the large number of customers using LogLogic, the mission critical nature of the business problems being addressed, and the system complexity that is introduced due to the previous topics, usability is a critical factor for LogLogic. Our goal is to strike an appropriate balance between ease of use and complex functionality. Let’s face it. Using LogLogic to handle forensics, or compliance, or general reporting and alerting is an easy solution to a very complex problem. We are taking the chaos of a massive amount of log-traffic and we are taming it via the use of log forensics tools, summary reports, and real-time alerts.

When I was on the support team one of the biggest questions customers would call and ask after purchasing the LogLogic appliance was “Now what? What should I be looking for?” - and the answer LogLogic came up with was the compliance suites. The compliance suites are a phenomenal way of helping the customer hit the ground running. It’s simple. “Install this suite of search filters, alerts and custom reports. Then tailor it to your network. Now you’ve solved your compliance problems and/or government mandates.”

14. Compatibility/Maintainability: To minimize downtime as part of providing high availability, LogLogic must provide compatibility from one release to the next. Backward compatibility means that the when a new version of LogLogic  is installed, it continues to work with the previously installed customizations and other systems witch which it interacts such as source devices and remote authentication services.

 Forward compatibility means that when systems around the LogLogic are upgraded, the LogLogic must continue to function as it did before, even though the new software may support new log formats. When fixes are provided for LogLogic, downtime for installing the fix must be minimized by providing support for incremented updates such as patches and hot-fixes. LogLogic must also offer several solutions to handle updates such as automatic update via the web, or manual file updates due to the appliances being offline.

Thank you for reading my five piece foray into the Enterprise. Please feel free to email me your thoughts, comments, or post them below. Stay tuned to LogLogic as we journey where no other LMI vendor has gone before."

Posted by on October 31, 2007 in Innovation , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Poll: Why Do You Collect Logs?

The previous poll (vote here, live results here, analysis here) proved to be a success so the next one is here.

This time the question is: "Assuming that you centrally COLLECT system, network or security logs from their originating sources, what is THE MAIN reason for doing it?"

Vote on!

UPDATE 11/11/2007: results and analysis are posted here

Posted by on October 31, 2007 in Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Log Collection Poll Results

My poll on log collection have been a great success and now the results are in. Here is the link to the running totals as of now (the graphic snapshot from yesterday can be seen here). Let's review and discuss the findings after running it for slightly over a week.

• First, which of my expectations were NOT met? Well, I did expect that firewalls will be #1, not Linux/Unix servers. Admittedly, the difference is not so big, but I am impressed: Unix syslog still rocks the logging world :-)
• Second, the top source of collected logs is also the hardest to analyze due to its lack of structure. Nowadays I treat syslog from Unix/Linux as "broken English" and not as "data." It is a dog to parse (that is why we are trying to find something novel)
• Third, I was amazed that database logs were THAT high on the list. Wow! All the evangelizing seems to have worked out :-)
• Fourth, Windows server log collection is still in the dumps - but we need it! Go grab LASSO and dump those event logs into syslog without pesky agents. Easy!
• Firth, other Unix logs - what are those? We might never know what the respondents meant: still, I think that these are binary audit logs and other fine-grained audit logging. Indeed, many people starting to look at BSM audits and other "ugly ducklings" of logging.
• Sixth, web server logs are gold - everybody knows it. The poll confirms this as well: they are #2. Some fun analysis tips from me are coming soon.

Next poll coming soon! Thanks a lot for responding!

Possibly related posts:

• Top 11 Reasons to Look at Your Logs

• Top 11 Reasons to Collect and Preserve Computer Logs

• Database log management

 

Posted by on October 26, 2007 in Innovation , Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Simple Log-based User Profiling for Activity Monitoring

Recently I've been looking into detecting stolen/shared access account credentials. Now, how can you detect that? No NIDS will trigger, NIPS will let is pass, no unusual types of log records might ever by  produced (especially if only limited logging is enabled). And what raises the stakes is that this type of activity is not  only about "hacked" accounts, but also about insider abuse of accounts.

However, there will likely be changes in how normal log records are produced.

Let's summarize some known methods for using a simple user "profile" to detect account theft aka account sharing aka user impersonation aka access with stolen/shared credentials. It implies that we've been collecting the logs before the incident and have a solid trail of normal users and legitimate account owners.

1. Unusual login source IP (e.g. normal user always comes from 10.0.X.Y or 10.1.X.Y, but now we see a login from 172.16.0.Z) - will work in some cases, but not for the free-for-all servers such as at a University
2. Unusual login time (e.g. normal user always comes from 9AM to 5PM, now we see 3AM) - will work in most cases, but will fail if the attacker happens to be in the same time zone
3. Unusual login session length  (e.g. normal user always stays logged in for 5-20 minutes, now we see a 10 hour-long session) - works only if logout is logged; might not catch a lot of realistic but malicious connections
4. Unusual login frequency (e.g. legitimate user logs in once a day in the morning, now we see dozens of connections) - will work for some cases, but others will be missed
5. Unusual login failure/success ratio before a successful login (e.g. normal user always types the password right the first time, now we see failures than successes) - not too reliable, obviously
6. Unusual list of user actions performed (normal user only reads these files, but now we see writes to a very different set of files) - will work most frequently, but needs more granular logging of file access, object changes, etc (audit logging) [more on this in the near future!]

So, if you have logs of user activities, at the very least, logins and logouts ( but having records of more user activities is always better!),  for the last few weeks or months, one can compute the above profiles using historical data and then compare them with current numbers (very similar to some of the methods from my classic log mining presentation).

The final missing bit is for how long to collect your normal user behaviors: I discovered that 1 week to 1 month works pretty well. Less time yields unstable results and more time necessitates much more data crunching without much gain.

Posted by on October 12, 2007 in Innovation , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Just What is Enterprise Class? Part IV

Here is the next enlightening post from Dimitri McKay, our super-brilliant network and systems engineer from the East Coast, where he continues to discuss the meaning of the phrase "enterprise-class," which is certainly MUCH more than a marketing buzzword! Part I is here, Part II is here, Part III is here. 

"10. Serviceability/Manageability: The scale of deployment of a LogLogic solution, whether it be managed remotely or locally, requires features which allow the solution to be managed effectively. FCAPS is an acronym which is often used to remember the following guidelines of  security log appliance management:

Fault, Configuration, Accounting, Performance, and Security

The users of LogLogic must be able to monitor what is happening on the system itself, as well as the network around it and applications on it, and to have the ability to diagnose faults when they occur.

Fault diagnosis requires the ability to collect sufficient information about the fault when it occurs, preferably without having to reproduce the fault. We log our own appliance, and anything else on the network, and then alerts can be handled via SNMP traps sent to an SNMP trap receiver such as HP Open View or IBM Tivoli. The other option is an alert sent to a remote pager or mobile device, or even an email to the NOC/SOC personnel.

11.  Customizability/Flexibility/Integrability: LogLogic is used to solve complex business problems on a large scale. For some it’s used for various compliance needs, such as PCI or SOX. To others, it’s an alerting or filtering tool, while forwarding a few of the log records on to a SEIM (typically, as much as it can handle, which is usually not as much as needed ...) or a specific other security tool. To others, LogLogic is used for general reporting, incident investigations or forensics. Regardless of how it’s used, LogLogic a different tool to different people.

LogLogic appliances are rarely rolled out as a single unit in an environment. Generally they are rolled out by location, by message per second (MPS) requirements... or by long term storage requirements, but any way you shake it, LogLogic architecture is designed to meet the needs of the Enterprise (and this is not to say that SMBs won't benefit from log management!).

Usually you’ll see several LX reporting/alerting appliances feeding back to a single long term storage ST appliance, but that’s not written in stone. Sometimes customers send all of their log data direct to the ST storage appliance (which handles a massive 75,000 messages per second) in order to take advantage of a single IP address destination. This, in my own humble opinion, is a good data center solution.

My point is, the architecture of the LogLogic appliances is a variable (but still easy!) which gives you total control. There is no firm “config” that is required for boxes to be placed in. Instead, the architecture is flexible, and allows for multiple configurations depending on the environment they reside in. This is the ability to remain agile.

Living in an enterprise world, we must adopt to new technologies as they become standards. Often some of the features on the LogLogic appliances overlap with other technologies already in use. Because of that, we have engaged the ability to utilize those technologies in those situations. One example is that LogLogic has but doesn’t always provide its own identity management functions. Typically there is another authentication management that the enterprise is already using, such as TACACS+ or RADIUS. The enterprise may have adopted particular standards to manage user databases and access control. For this LogLogic needs to be flexible in it’s architecture, allow customizability within the enterprise, and integrate well within the framework of the already existing environment.           

12. Support: With any mission critical software deployment, the enterprise must have a reliable way of getting support for diagnosing problems and getting fixes and advanced replacements. This includes 24x7 availability for support personnel and an organization with the experience and expertise to understand how the appliances are used in enterprises. Our support is not only top-notch, but is also praised by our customers!

Thank you for tuning in to part lV of “Enterprise Class” where I’ve laid down WHAT enterprise-class really is, and how LogLogic has tackled it. I’d also like to thank our competitors who have been visiting my blog. You can imitate, but you’ll never duplicate us, boys!

Next piece will conclude the series ... stand by!"

Posted by on October 11, 2007 in Innovation , Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

PoS Logs out of PCI Scope? Surely You Are Joking...

... well, turns out they were dead serious. As I expressed my puzzlement to our resident PCI auditor, he explained that PoS logs and overall security of PoS devices are often "in-scope for PCI, but out of scope for a typical PCI audit."  How bizarre is that?  But let's start from the beginning.

First, what on Earth is a PoS? PoS, or Point-of-Sale terminal, is a machine that stores (or whoever else who takes credit cards) use to process credit card transactions: scan cards, communicate with verification server, print receipts, etc. It might be standalone or combined with a cash register. It might very very simple - just card reader + transaction unit in a single hardware unit - or as complex as a Windows PC with a cash drawer and no software updates (a scary thing indeed!)

So, in the latter case, there are certainly logs involved. In fact, there are also PoS-specific application logs, such as this example below, coming from an IBM SurePoS device:

--------------------------------------------------------------------------------------------------------------------------------

01/11 09:11 CC     5 W518 PROGRAM ADDDDDXUXDL HAS ENDED                           

                   B3/S111/E007 REASON=2 TYPE=3 RC=00000000            

SOURCE: OCF                                                                     

REASON: Application ended            PROGRAM TYPE: Background               

RC: No error      

--------------------------------------------------------------------------------

PoS devices might be configured to store credit card numbers locally (for backup) and also to offload them to a "branch server" (a store server or both a store server and a regional server). Are there logs of who accessed them on the local PoS system? Unlikely. Are they looked at? Probably not.  Maybe the logging is done better on the branch or store central server, but even this is not a certainty.

Overall, I am willing to bet a bottle of decent champagne that very few people, if anybody, in the whole world are regularly looking at PoS logs. At some happy point in the future, I predict they will start since the Beast of PCI will make them :-) When this happens, we will talk about PoS log analysis.

As of today, you would do comparatively well if you will collect and save them and thus will have a chance to review them for incident response for your next data theft case (or show them to an unusually nosey PCI auditor...)

More fun PoS security reading is here [PDF].

 

Posted by on October 05, 2007 in Compliance , Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

Just What is Enterprise Class? Part III

Here is the next enlightening post from Dimitri McKay, our brilliant network and systems engineer from the East Coast, where he continues to discuss the meaning of the phrase "enterprise-class," which is certainly MUCH more than a marketing buzzword! Part I is here, go read it first. Part II is here, go read it next.

"This is the third piece in the five part series titled “Enterprise Class”. Here we’ve gone through what IS enterprise class and how that corresponds to the LogLogic solution. Feel free to post your comments.

 7.Performance: Performance is a broad topic. We’re not just talking local log collection performance, but also the ability to perform efficiently while in a distributed environment is critical to enterprise class hardware and software.

While the ability to add more LogLogic appliances allows more devices to be logged and application data to be collected, if the software does not perform with a high level of efficiency, the cost to deploy a system for the required number of devices will be prohibitive.

LogLogic’s top end reporting appliances handle 4000 messages per second sustained. That is a HUGE amount of traffic. if you’ve ever had to sift through it, you’d agree. Think about it. That’s 240,000 messages per minute. 14.4 million messages per hour. That’s 345.6 million messages per day. Grep that with your syslog server. Parse that with your SIEM. It’s not going to happen (not without an ENORMOUS price tag). And our log collection appliance do even more, way more: 75,000 messages  per second sustained. Wow!

Performance on a LogLogic appliance is measured based on several factors: CPU usage, messages per second, and disk usage which are important not only for the main functions of the system such as receiving and parsing logs, but also for the ability to run reports and searches on those messages.

8.Security: Because LogLogic is used for mission critical functions, security is essential. LogLogic must ensure that users who are authenticated to the appliances only have access to the functions they need and access to the log sources that are determined by their job role. LogLogic must also offer the ability to encrypt data as it moves between appliances (which we do via LogLogic TCP) and to also offer the ability to use WORM or data at rest encryption (which is handled by the EMC Centera or NetApp for WORM and Decru for data" at-rest" encryption).

LogLogic also covers security with the ability to verify that the system is secure through audit trails, general protection of the appliance through a secure linux kernel locked down in the default install, perform self logging, and then hashing the raw logs which are kept in an immutable format. At LogLogic we take security serious, and the box is put through a variety of vulnerability assessments on a regular basis.

9.Documentation: Because LogLogic handles device support through a specialized group of folks we call “LogLabs” we are able to add devices at an expeditious rate. With each of those new devices comes the required measures to configure each of those devices for logging, and with that knowledge transfer that to documentation. Over time our Documentation Group has continued to grow. Knowledge management is essential to managing the enterprise class solution while alleviating the strain on any of the support or field engineering staff.

Check back soon, to see more about how LogLogic does Enterprise right. In the next episode I will run off on a tangent about usability and flexibility/interoperability/customizability and support! We are boldly going where no other LMI vendor has gone before!"

Posted by on October 04, 2007 in Innovation , Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

Log Trustworthiness Hierarchy

This post is inspired by the old TaoSecurity post Enterprise Trust Pyramid where he explains that he trusts some security, system and network evidence data more than other. For example, dedicated NSM sensor records are trusted more than vanilla server logs. With this post, I am looking to establish a log trustworthiness hierarchy so that people start thinking about trusting log data as a kind of a spectrum, from "probably trash" to "guaranteed to be an accurate record of activities."

So, do you trust your logs to accurately depict what happened on the system or network? Which logs do you trust the most? How do we increase this trust?

My first draft of such trust hierarchy follows below (from low trust to high trust):

1. Compromised system logs (mostly trash, but might contain bits that attacker missed/ignored)
2. Desktop / laptop OS and application logs (possibly changed by users, legitimate systems owners, etc)
3. All logs from others systems where 'root'/Administrator access is not controlled (e.g. test servers, etc)
4. Unix application logs  (file-based)
5. Local Windows application logs
6. Local Unix OS syslogs
7. Unix kernel audit logs, process accounting records
8. Local Windows server OS (a little harder to change)
9. Database logs (more trusted since DBA cannot touch them, while 'root' can)
10. Other security appliance logs (located on security appliances)
11. Various systems logs centralized to a syslog server
12. Network device and firewall logs (centralized to syslog server)
13. Logs centralized to a log management system via a real-time feed (obviously, transport encryption adds even more trust)

Admittedly, the differences between some of them are minor or even non-existent ...

To conclude, some logs DO in fact provide reliable evidence in case of an incident; you just need to know which ones to trust and which ones to only consider to be "hints" (or possibly even a misdirection). But of course, you need to first have logs and then look at them.

Posted by on September 27, 2007 in Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Just What is Enterprise Class? Part II

Here is the next enlightening post from Dimitri McKay, our network and systems engineer from the East Coast, where he continues to discuss the meaning of the phrase "enterprise-class," which is certainly MUCH more than a marketing buzzword! Part I is here, go read it first.

"4.Scalability: To support a large number of devices in growing enterprises, SIEM and log management products differ a great deal in how they scale. Some of the SIEM products require you to add as many as three new appliances, (which all carry a hefty price tag) just for a slight increase in throughput. Others require you to forklift out your existing hardware to bring in new, bigger (and more expensive) equipment. In my own humble opinion these options are completely ridiculous.

The LogLogic scalability approach is uniquely simple. “Add log-traffic to your LogLogic appliances until you reach capacity on that box. Then add another box. Rinse and repeat. Each box does it’s own collection, does its own parsing, reporting and searching, and does its own storage. No special other boxes required. All encompassing solution.

Of course, with “Scalability” comes other requirements of how to add capacity not only vertically, but what of horizontal scalability? Such features would include the ability to report across all LogLogic appliances, manage users and their access rights across a global environment, and use some form of remote authentication such as TACACS+ or RADIUS. This distribution should be an asset, and not a limitation of the overall system scalability. All of these are addressed and in the current LogLogic 4.x.

5.High Availability: Due to the number of corporations and government agencies depending on LogLogic for regulatory compliance, system security and forensics, as well as heartbeat monitoring and troubleshooting, it’s safe to say that LogLogic is “Mission Critical”.

Because LogLogic is a “Mission Critical” system, it must support the internal and external capacity for zero downtime via high availability. This is accomplished via bonded NIC’s, hot-swap drives in various RAID arrays and Mirrored sets, hot swap redundant power supplies and most importantly... the ability to link the boxes together in an active pair. Should there be a failure in a LogLogic pair, both appliances are exact clones of each other, and the passive appliance jumps right into gear as the primary. In milliseconds. It’s genius.

In a perfect world, there would be zero downtime on “Mission Critical” systems. Here at LogLogic, we try to minimize that to as little as possible through the use of various forms of HA, and it shows. Have you seen our new hardware?

6.Reliability: Although high availability plays a part here, I prefer to think of reliability as the quality of the code-base, the product design itself, and the forward motion of feature sets and additional device support in a timely fashion.

LogLogic is built on a stripped down lean, mean hardened Linux platform. From there we didn’t go the route of a proprietary datastore which can start as a good idea and then turn into a huge hulking mess with nothing but frustrations stemming from using, maintaining and increasing the capabilities of, but rather we went with a solid database solution for storing meta-data. Industry standard. Well known. Easy to use. Robust.

Then we built in some brilliant parsers and a very straight forward GUI. Sprinkle that with some great features and a little bit of magic and you have a LogLogic appliance. Our upgrade plans and implementation are fast and furious. We release new device support monthly. Monthly additions. Monthly updates.

At the end of the day, however, extenuating circumstances happen. Should there be a hardware or software problem, LogLogic support is there to help. With 24/7 support and advanced replacement available to our platinum level enterprise class customers, all the bases are covered.

Build the appliance on a solid kernel, with a well known and well tuned database. Add HA, some of the finest engineering talent in the world, and a great support team, and LogLogic does enterprise right.

Stay tuned for the 3rd installment of “Enterprise Class”. LogLogic is going where no LMI vendor has gone before."

Posted by on September 27, 2007 in Innovation , Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

More Web Proxy Log Tips

After publishing my proxy log tip (here), I figured I'd post a few more mini-tips on web proxy logging as well as a link to my full presentation (webcast with voice, slides only) on web proxy log management.

First, why look at proxy logs? Apart from my overall answer that applies to all logs, proxy-specific reasons are the following:

1. Review users’ activities on the web (not just surfing!)
2. Monitor applications' HTTP activity
3. Detect Web-enabled malware traffic
4. Study proxy performance metrics

Most people just focus on #1 above and kind of forget #2-#4. Also, the focus of #1 is often narrow - what do they surf at work?- and not broad - what do they do on the web? - which is much more useful. While there is no direct mention of proxy logs in recent regulations, monitoring what users do with YOUR data is clearly part of the compliance mandates (and, obviously, a good idea in general!) Indirect references to proxy logging can be seen in the following:

• Proxy monitoring is part of the overall control and governance (thus SOX, HIPAA, GLBA, etc)
• Legal requirements to have audit trails (thus HIPAA, PCI)
• Breach disclosure laws also impact (SB1386 and others, soon US-wide)
• On the flip side, privacy laws might mandate the opposite i.e. NOT having logs.

So, please treat proxy logs with the respect they deserve!  Here is my full presentation (webcast with voice, slides only), on analyzing and managing web proxy logs.

Related posts:

• Anton Logging Tip of the Day #12: Proxy Log Fun - Proxy Logs vs Information Leakage

Posted by on September 24, 2007 in Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Just What is Enterprise Class? Part I

Here is another enlightening post from Dimitri McKay, our network and systems engineer from the East Coast, where he discusses the meaning of the phrase "enterprise-class" (which is sometimes known to be abused in marketing literature...)

"The Enterprise is generally a huge task, and this post itself is quite large. Due to the mass of both this post and the average Enterprise roll-out, I will break this down into 5 parts. Here is part 1 in the 5 part series.

I hear the term “Enterprise Class” as a buzz word to define hardware and software. Sometimes it’s used to describe a feature set, and to some it’s a scalability reference. Here’s what “Enterprise Class” means to me, and how it pertains to LogLogic solutions as a whole.

1. Multi-user: LogLogic handles the Multi-user process in a two prong attack. First, we offer the basic granular local account system which gives users the rights needed to do the job on only the devices they require. Now, on the enterprise side, we have added the ability to connect to TACACS+, RADIUS and soon LDAP/AD servers in order to alleviate the task of managing users on the LogLogic Appliances. Multi-user is a must for anything that is to be considered “Enterprise Class”. I’m hoping to see some additional authentication methods in the future, however, as for now the TACACS+ and RADIUS option certainly covers the majority of our enterprise customers.

2. Accessibility: In 1998, Congress amended the Rehabilitation Act to require Federal agencies to make their electronic and information technology accessible to people with disabilities. Inaccessible technology interferes with an individual's ability to obtain and use information quickly and easily.  Section 508 was enacted to eliminate barriers in information technology, to make available new opportunities for people with disabilities, and to encourage development of technologies that will help achieve these goals. Most of the specifications for software pertain to usability for people with vision impairments. For example, one provision requires alternative keyboard navigation, which is essential for people with vision impairments who cannot rely on pointing devices, such as a mouse. Other provisions address animated displays, color and contrast settings, flash rate, and electronic forms, among others (see more at Section508.gov).   Now, at LogLogic, we try to make our GUI as straight forward and easy as possible. We adhere our GUI to the standards and framework put forth by the web browser itself, which thereby adheres itself to the accessibility options within both that browser and also the Operating System.

3. Localization: As with any business solution, our world isn’t set in one language or specific dialect. As a whole, LogLogic offers language options not only in English but also in languages relevant to foreign countries that are bound by the same requirements that LogLogic can address such as JCobit and J-Sox, which are the Japanese versions of COBIT and Sarbanes-Oxley (SOX) compliance. As we move forward LogLogic continues to add alternative language support such as Korean and Chinese.

Stay tuned for the next episode of “Enterprise Class” where I will run off on a tangent about HA, reliability and scalability."

Posted by on September 24, 2007 in Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

More Project LASSO Updates

Dimitri McKay, our network and systems engineer from the East Coast contributes this fun story: "When I first started at Loglogic, the only option for retrieving Windows logs was via an open source product called Snare. Snare was a simple concept. It would basically tail a Windows event viewer and whenever a new event was written, Snare would convert that to syslog, and send it over the wire (UDP or TCP) to its destination. Basic concept, basic execution. 

    Now, all of this was well and good, however, I couldn’t help but feel like an agent was not the greatest of solutions. As a former Windows Engineer, I was sort of put off with the idea of agents as a whole. It seemed like everyone was offering agents, and those agents were never happy on my systems in some form or another. So began Project Lasso. 

    Over some Thai food, Matt Foley, Andrew Morris and myself had a conversation about an agent-less Windows solution, and later that day I found myself writing a PRD for the “Windows Remote Event Collector” which was code named Project Lasso. 

    The name stuck, and with our 4.0 release I’m really happy with the ongoing development of the product. The concept is simple... Lasso is installed either as an agent to monitor itself, or on a “Project Lasso Server” where it uses WMI connections to connect to the other hosts it will be pulling logs from. There it pulls the string dll’s to a local repository, and then pulls the windows events themselves. Both the string dll’s and the events are then sent to the receiving syslog or Loglogic appliance where they are parsed and indexed. [Anton: they can also be sent to any other syslog receiver, such as a syslog-ng server]

    In 4.0 we added a few features I’m very excited about. 

The ability to use custom shares. In version 3.x we needed a Domain Admin account to pull the full Windows events. The Domain Admin account was the only account which had access to:

• The Event Viewer (for the header and events themselves)
• The Registry (to point to the string dll’s.) 
• The file system (via c$ to get the string dll’s.)

In Project Lasso 3 if you set some Active Directory policies, you could eliminate the need for a Domain Admin for the registry and event viewer, but not the C$ admin share. The only other account that had access to that was the Backup Administrator. Now, in Project Lasso 4 we don’t have that issue any longer because we can configure just a standard Lasso User account to have access to the registry, the event viewer, and a local drive via a custom share.

The ability to change the Hostlist.ini on the fly: In Project Lasso version 3.x when you added hosts to be monitored, you had to restart the Lasso service. Well, this process became somewhat onerous in that every time an Administrator needed to add new servers/clients to be monitored by Project Lasso he had to re-start the service which would take quite a bit of time to check those string .dll’s for changes or additions and then start pushing logs to the destination server. This could sometimes take more than a few minutes depending on the number of hosts. So much for real-time alerting or reporting. None of this is a problem anymore as the Project Lasso service will now grab the new hosts on the next pass.

Shared DLL Repository: Prior to Project Lasso version 4, when a dll Repository was created on the Lasso Server, it downloaded all .dll’s for all monitored hosts. This means there was a ton of the same .dll’s in the repository as each machine would most likely have the same .dll’s. These folders were about 120MB large in Lasso 3 times the number of hosts you were monitoring. That can be a big storage problem in a short amount of time. In Lasso 4, the dll’s are shared among hosts so it has a much smaller disk space requirement.

In closing, if you’re interested in routing Windows Events to a syslog or Loglogic appliance, you can do so via Project Lasso either in agent mode or in agent-less / remote collector mode.

Feel free to check out the always free Lasso 4 on Logforge"

Indeed, Project Lasso  is in wide use among LogLogic customers as well as in the broader world. Deal with Windows logs? Grab Project Lasso!

Posted by on September 17, 2007 in Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

Read "PCI Compliance" book chapter on logging!

As you know, the "PCI Compliance" book is out. Syngress/Elsevier publisher released one chapter from the book for free: and it is my chapter on log management and logging in PCI! Enjoy it here! [PDF] The book (and, in fact my logging chapter!) already got some glowing reviews.

While people still argue whether PCI is simple or overly complex, PCI DSS is certainly motivating a lot of people to take a long hard look at their IT security ... For example, more people are starting to look at database logs as a result of PCI. While some vendors are still lagging behind, you can get your database logs into LogLogic today!

Posted by on August 24, 2007 in Compliance , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

Logging Glossary: Log Preprocessing

As I mentioned here,  I started publishing the LogLogic Logging Glossary. So, here is the eleventh term (first second third fourth fifth sixth seventh eighth ninth tenth):

Log Preprocessing

1. The process of modifying or transforming a message as preparation for later stage processing.

Preprocessing can include multi-line message handling, encoding conversion, and special character substitution.

2. The prerequisite analysis of headers or other information in a log that define the formats or values present in the log messages.

The preprocessing assures correct parsing and identification of values. 

Posted by on August 20, 2007 in Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

Just What is "Scalability"?

Remember that discussion about syslog servers? While some people might try to sell you a mere syslog server for big money, others fall in the opposite end of the spectrum: they think that building a high-performance scalable log management system is as easy as installing a syslog on a machine ...

This fun story was related to me by Dimitri McKay, our network and systems engineer from the East Coast. He said: "Our biggest competitor in the field seems to be home-grown syslog servers. Although tons can be saved creating a generic "syslog server," at the end of the day, the resources and man-hours needed to maintain the thing offset the savings to total cost of ownership. I've seen several cases where a customer has a number of these servers running, 'grepping' their way through mountains of log data, and setting up script after script after script. Usually such a thing is started by someone very clever on that team, who gets bored, and moves on to something else, or some other job, and the syslog servers fall into disarray because nobody knows how and what to do with them.

When comparing an open source syslog server to a high-performance (75,000 log messages/second!) Loglogic ST, you do get what you pay for.

Specifically, one of our government customers hired a new employee who looked at the Loglogic ST3010 and expressed his disdain for commercial "out of the box" solutions. His exact words? "I can totally build that for like nothin'." So off our new friend goes, putting together a home-grown syslog server. 

First, he starts with a Linux kernel, installs a syslog collector, and then opens the flood gates direct from the Loglogic ST3010 via log message routing. The syslog server suddenly becomes unreachable, non-responsive to pings, SSH attempts, the whole nine. The flood gate of log traffic was then turned off.  Back to the drawing board.

Our new employee then comes back with two machines taped together in a cluster. The flood gates are opened, forwarding messages direct from the ST3010 to the home-grown solution, and the same result ensues. No response from pings, SSH, or even console access.

This continues on... three servers clustered, four servers clustered, five servers clustered... till the sixth server was added, and that's when our customer pulled the plug on his new hire. Too much time and hardware resources were being used in a project that was being done quickly and efficiently by one Loglogic ST3010 appliance. Why try to fix it, if it “ain’t broken!”

At the end of the day, you get what you pay for, and you pay for what you get."

Posted by on August 10, 2007 in Innovation , Log Management & Intelligence , LogMatters , Security | Permalink | Comments (0)

Anton Logging Tip of the Day #12: Proxy Log Fun - Proxy Log Analysis for Possible Information Leakage Detection

Following the new tradition of posting a tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative. One of the bloggers called it "pay it forward" to the community.

So, Anton Logging Tip of the Day #12: Proxy Log Fun - Proxy Log Analysis for Possible Information Leakage Detection

You probably know that web proxies (such as Squid, BlueCoat SG and others) produce a lot of detailed logs, that record all web traffic flowing through the proxy as well as pass/block decisions made by the proxy's content filters and possibly embedded anti-malware tools. Proxy logs can be used for a whole range of things, from routine monitoring for Acceptable Use Policy (AUP) compliance to malware detection as well as possibly looking for security scourge of 2007 - web browser attacks by malicious or compromised web servers.

Specifically, in this tip we will learn how proxy logs can be used for detection of file uploads and other outbound information transfers vie the web. First, think what is the legitimate use of file upload functionality in your environment. For example, if using web-based mail services is allowed, then sending an attachment will include an upload. What else? The rest will be considered at least suspicious...

In addition to file uploads, some malicious or commonly unauthorized applications will use similar methods to steal or transfer data, that will be reflected in proxy logs. Looking for HTTP methods (such as POST) and content-type in combination with either known suspicious URL  or user-agent (i.e. web client type) can often reveal spyware infections, actively collecting data. Admittedly, a well-written spyware can certainly fake the user-agent field so it is clearly not reliable, but still useful to add to our query above. 

So, here are some of the criteria we will use to look for information uploads in Squid and BlueCoat SG proxy logs:

• HTTP method (logged as "cs-method" by BlueCoat) = POST  (as opposed to the usual GET, used to retrieve web content).
• For information uploads: content type (logged as "RS(content-type)" by BlueCoat) = pretty much anything but "html/text" (which is the type used for uploading  web form contents) - especially try content types  "application/octet-stream", "application/msword", "application/powerpoint", "application/vnd.ms-excel", "application/pdf" and a few others to look for common file uploads.
• For spyware and application data transfers: user-agent set to anything but the common ones (i.e. not Mozilla, iTunes, LiveUpdate, etc) or even to "unknown." One can also try user-agent containing your favorite messaging app (e.g. "MSN Messenger", etc)  and see which such applications are in use.

(if you feel adventurous, other interesting content-types to try are "application/x-javascript" and "text/javascript")

Here are the examples found in proxy logs using the above query, including some "classics" (while spyware specimen are a bit dated, this method of  detecting them via logs is still relevant and useful):

1. 1124376766.026 RELEASE -1 FFFFFFFF 4734C557F9315105CA6BE0FA56B94D55 200 1124276674 -1 -1 unknown -1/0 POST http://reports.hotbar.com/reports/hotbar/4.0/HbRpt.dll
2. 1124392388.975 RELEASE -1 FFFFFFFF 810FFBF233584C330353CF0A8C31F5D2 503 -1 -1 -1 unknown -1/813 POST http://log.cc.cometsystems.com/dss/cc.2_0_0.report_u
3. 2007-05-19 03:55:12 160 10.1.1.3 - - - OBSERVED "Spyware/Malware Sources;Spyware Effects;Web Advertisements" - 200 TCP_NC_MISS POST text/html;%20charset=utf-8 http bis.180solutions.com 80 /versionconfig.aspx ?did=5342&ver=1.0 aspx - 10.1.1.2 273 175 - - none - -
4. 2007-05-21 03:10:40 4 10.1.1.3 Joanna- authentication_redirect_to_virtual_host PROXIED "Search Engines/Portals" - 307 TCP_AUTH_REDIRECT POST - http storage.msn.com 80 /storageservice/schematizedstore.asmx - asmx "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; MSN Messenger 7.5.0324)" 10.1.1.2 791 2566 - - none - -
5. 2007-05-22 21:35:09 215 10.1.2.237 200 TCP_NC_MISS 217 8122 POST http kenobi.example.com /exchange/john.smith/Drafts1/RE:%2520CustomerList.xls-2.EML - - DIRECT kenobi.example.com - "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" PROXIED none - 10.1.170.42 SG-HTTP-Service - none –

The first three are traces of spyware (one was even identified by a BlueCoat content filter as "Spyware/Malware", the fourth is MSN Messenger-based activity while the fifth is emailing the Excel file via web mail.

Here are some other signs that will make the above log entry extra-suspicious is:

• A dead giveaway: upload happens to a "known bad" URL (e.g containing "gator" and others above) 
• Upload happens to an unresolved IP address (do a "whois" on it!)
• Uploads happens to a port not equal to 80 (i.e. the URL contains a port such as http://10.1.10.10:31337)
• Upload has confidential file name in the log entry (e.g. somebody dumb emailing a sensitive file to himself - as discussed here)

Overall, this log analysis method is good for casting a broad net to catch not just spyware-infected systems, but also unauthorized applications (e.g. method=POST and user-agent=iTunes), instant messaging (e.g. method=POST  and then by user-agent, content or URL), simple forms of data theft and document handling policy violations (emailing files to self via web mail: method=POST and sensitive file name present in the entry; also content-type set to popular Office file types) as well as other abuses of web access. As a result, proxy logs provide an extremely rich AND readily available source of data about threats that users face!

To top it off, one promising direction of future research is using web proxy logs to detect client-side exploits by malicious web servers (more on this in the near future!)

Possibly related posts:

• Anton Security Tip of the Day #6: The Other Web Log
• All my past security and logging tips

Also, I am tagging all the tips on my del.icio.us feed. Here is the link: All Tips of the Day.

Posted by on August 07, 2007 in Innovation , Log Management & Intelligence , LogEd , LogMatters , Security | Permalink | Comments (0)

A 'Syslog Server' for $X0,000? No!

I hate it when people call what we provide (i.e. log management) a "syslog server." I really do. Why will someone pay $X0,000 for just a box to "collect syslog?" No, really, why? I won't! It does indeed sound silly and wasteful.
By now, many people understand that log management is not about collecting syslog in one big trash can. You can do that much easier and cheaper if that is indeed your goal. Why would someone collect syslog in a trash can is a separate story :-), even though collecting logs is pretty useful at times. But using the collected log data is much more valuable!
So, please get it! Log management is about scalable (meaning you can deal with a lot of data) collection (yes, collection too) + retention (meaning storage and then destruction) + analysis (real-time and historical methods of making sense of data) of all types of log data (not just syslog!!!), and about making such data available for all organizational needs (security, compliance, operations, fun bed-time reading :-), etc).

Posted by on July 09, 2007 in Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Top 11 Reasons to Look at Your Logs

As promised, I am following my Top 11 Reasons to Collect and Preserve Computer Logs with just as humorous and hopefully no less insightful "Top 11 Reasons to Look at Your Logs." 

1. The first reason is again disarmingly simple (is it, really? :-)). Read PCI DSS lately? Glanced at HIPAA? Suffer under FISMA? Yes, all of the above regulations say that you must not only have, but also review logs periodically.
2. Are your servers compromised now? How do you know if all your logs are stashed on a tape in a closet? Look at them! Now!
3. An incident happens. Really, who needs extra motivation to look at logs in such case? Using logs for incident response is a true "no-brainer" (however, you need to be pretty "brainy" to actually analyze them in case of an incident)
4. Users - from a CEO to a janitor. You do have to know what they do on your IT systems! How? Read the logs! Everybody leaves tracks.
5. Systems log plenty of errors. Sometimes they are silly, sometimes - benign. However, often they mean that "stuff" is about to hit the fan. Periodic review of logs reveals them and saves the day.
6. Network slowed to a crawl?  Applications are slooow? Server is not ... well, serving? :-) Where is the answer? In the logs, but you need to read them and understand them.
7. That policy you wrote a few months ago. Anybody following that? Anybody remembers that? Halloooo! Check the logs and you'd know.
8. By now you know that your auditor might ask for your logs. But do you know they might also check whether you looked at them? Do you? Review the logs and leave the record of this activity in the logs.
9. Change can be good. But then again, it may be the sign that your controls are lacking. Who changes what and when? From what and to what? Just review the logs.
10. Now, you hate looking at logs. You have too many (as if everybody else doesn't...)! In this case, look at a specific subset of logs that you never saw before- NBS. Or just deploy log management that can do it for you.
11. Logs can help you predict the future (if you review, know and love them :-)). Don't believe it? If you read them for long enough, you develop an ability to predict the future, albeit mostly future problems :-)

See also: Top 11 Reasons to Collect and Preserve Computer Logs. 

Coming soon: "Top 11 Reasons to Secure and Protect Your Logs"!

Posted by on July 06, 2007 in Log Management & Intelligence , LogMatters , Security | Permalink | Comments (0)

PCI Book On the Way...

Anton Chuvakin, a Loggie, has a co-authored book on the way on PCI Compliance:

I am sure that "everybody in the know" is already, well, in the know, but still - here it comes, the first book on PCI: '"PCI Compliance: Understand and Implement Effective PCI Data Security Standard Compliance"by Tony Bradley (Author), Anton Chuvakin (Author), Anatoly Elberg (Author), Brian J Koerner (Author)'.

Posted by on March 22, 2007 in LogMatters | Permalink | Comments (0)

IT Security's Top 59 Influencers Includes LogLogic Vets

Our own Anton Chuvakin was named to IT Security's most influential security experts of 2007.

He is in some very good company....check the site's complete list of the most influential security experts of 2007 - from corporate tech officers and government security types, to white hat hackers and bloggers.

Joining Anton is LogLogic's CMO Andy Lark, named in IT Security's Bloggers List.

The list certainly generated a lot of buzz in the security blogosphere.....

Technorati : Log Management

Posted by on March 20, 2007 in LogMatters , Security | Permalink | Comments (0)

Links & Good Reading

Jon Olstik touches on the good news and bad news for security management:

"What used to be a small, tactical security event management implementation is evolving into a much bigger opportunity. As enterprise organizations collect log and network flow data, they want to provide it to the network operations, compliance management, system administrators, lines of businesses and security for all kinds of analysis. In geeky technical terms, what was a security-focused data mart is turning into an enterprise IT data warehouse for all kinds of data analysis, event monitoring and reporting. "

Exeactly. Watch for more on Log Data Warehousing!

Posted by on March 07, 2007 in LogMatters | Permalink | Comments (0)

Expensive Sox...

SOX spending accounts for 20% of global governance. From over at Findtechinsights:

Sarbanes-Oxley spending isn't likely to change much this year, according to AMR Research. The company's latest report predicts that, despite recent revisions to relax the corporate reform law's requirements, 2007 Sarbanes-Oxley compliance spending will stay at $6 billion and account for around 20 percent of the total governance, risk and compliance spend of $29.9 billion. The latter figure represents an 8.5 percent increase from 2006. The numbers don't change much, explains AMR Research VP John Hagerty, because some small companies will be spending on Sarbanes-Oxley compliance for the first time in 2007.

Posted by on March 01, 2007 in LogMatters | Permalink | Comments (0)

Research: Don't let sleeping logs lie

Who is looking at your logs? Better yet, what systems and policies do you have in place to monitor log files?

Datacenters are increasingly turning to log management to ease compliance burdens, and gain instant insight into the network. Nemertes Research says that "almost 80% of large and small enterprises have a data center-specific security policy defined, and of those with policies, more than 80% regularly test compliance with them."

The bad news?

Although everybody engages in some level of system logging (whether solely for security reasons, or in support of regulatory-compliance efforts as well), fewer than 30% of companies log all systems, and fewer still collect the logs at a central location for review and analysis. In fact, most logs are left in place and never reviewed except in the heat of a crisis, or worse, in the aftermath.

At NetworkWorld, Nemertes offers some suggestions on log management and the security considerations for customers, with recommendations of what to look to vendors to deliver in a strategy.

Not knowing what is going on in your enterprise is not a good defense, and point products only serve to add to creating even more silos of information across the enterprise. Taking the Log platform approach, with SOA and other architectural considerations are how we at LogLogic approaches log management and intelligence, and one that our customers are trumpeting!

 

Posted by on February 20, 2007 in Compliance , Log Management & Intelligence , LogMatters , Security | Permalink | Comments (0)

Preventing data breaches is hard; detecting them later can be harder

Nice story in ComputerWorld that features LogLogic. Some quotes:

To quickly and consistently detect such intrusions, IT managers need to be able to collect and analyze literally every transaction flowing through their networks in real time, according to Maness. "You've got to know what every single packet on the network is doing, where it's coming from, where it's going and which ones are bad."

Some vendors such as LogLogic Inc. are beginning to offer more efficient ways to sift through voluminous log data and focus on the issues that matter, Maness said. Such products can complement security event management tools, he said.

LogLogic's hardware appliances are designed to automatically capture and store log data from firewalls, routers, servers, applications, operating systems and other devices, said Andy Lark, a spokesman for the San Jose-based company. The appliances can be configured to generate near-real-time alerts when the logs show violations of predefined polices, such as those associated with Payment Card Industry standards, he said.

Posted by on January 26, 2007 in LogMatters | Permalink | Comments (0)

NIST Expert Says Legacy Systems Must Be FISMA Compliant Within 1 Year

Legacy systems are expected to be in compliance with NIST within 1 year of the publication date, according to Dr. Ron Ross of NIST in his presentation today about FISMA Compliance. And that is not all! Systems under development are expected to to be in compliance immediately upon deployment.

We sponsored a Government Computer News webcast on FISMA Compliance this morning with Dr. Ross to record attendance! (Over 900 people signed up!)

Ross went through FISMA guidelines, cleared up misconceptions on FISMA Compliance, and made recommendations for sound strategies to to get compliant with FISMA in short order.

Three key takeaways:

"Successful FISMA Implementation demands that organizations adopt an "enterprise-wide" Security Strategy"

"Common controls must be continuously monitored with results shared with all information system owners"

"Continuous Monitoring; Facilitates annual FISMA reporting requirements"

Log Management and Intelligence delivers FISMA Compliance in minutes. Our just announced FISMA Compliance and Control Suite helps government agencies verify that information security policies are being followed, substantially reduce audit time and expense, and achieve FISMA compliance. Out-of-the-box reports and alerts directly map to NIST standards, including NIST 800-53 (security controls) and NIST 800-92 (log management), providing an efficient, easy-to-implement solution. Our approach is cost-effective, using all available log data to automate the process of auditing and enforcing policies - and supports 100% of all log-related IT controls as outlined by FISMA. And -- its the first FISMA compliance solution based on log management and intelligence.

Technorati : Compliance, FISMA, LMI, Log Management, LogLogic

Posted by on January 24, 2007 in Compliance , Log Management & Intelligence , LogMatters , Security | Permalink | Comments (0)

The Demystification of Event Logs

Will Kelly writes at processor.com that "Demystifying event logs requires proactivity with an eye toward retention, review, and automated tools to ensure that your log events are presented in a usable and actionable manner to your data center team."

LMI fulfills this charter. Kelly quotes our own Anton Chuvakin:

He also advises a proactive approach to handling log management vs. opening them when a network outage or security issue occurs. "Not looking at the logs until something happens is a big mistake," according to Chuvakin, because regular viewing of your logs enables you to see early signs of problems, such as security incidents like probes, not just trends.

Read the entire article here.

Technorati : Compliance, Event Management, LMI, Log Management, LogLogic, Logs

Posted by on January 22, 2007 in Compliance , Log Management & Intelligence , LogMatters , Risk Management | Permalink | Comments (0)

ROI of PCI compliance

Interesting post here on the ROI of PCI Compliance - along with a good primer on ROI. We've got a few comprehensive ROI models over at LogLogic.com. While some companies build comprehensive business cases for PCI compliance, others recognize that more than anything, an effective compliance program is about recognizing that the data customers entrust them with deserves protecting.

Mike gets at the issue more directly:

Whoever invented ROI should be beaten with a stick. Actually, it's an important concept for business management, but for security - it doesn't work so well. So I read with interest, this piece on ROI for PCI compliance. But it leaves me wanting. Wanting what? Basically, I want the discussion to just go away. The major benefit of compliance is in not getting hacked? That's ridiculous. The benefit of compliance is in making your auditors go away and ensuring you won't end up like our friends at TJX, all over the front page with your dirty laundry bared for all to see. It's not about offsetting fines, it's about protecting the contract you have with your customer to protect their data. Strong security will give you compliance. If you just try to buy compliance, you will end up with nothing but a big crisis communications bill.

There are other ways of getting a return on your PCI investment. The most recent being Visa's program, effective October 1, 2007 where merchants can qualify for lower interchange rates for being PCI compliant.

Posted by on January 22, 2007 in LogMatters | Permalink | Comments (0)

Anton's Security Tip of the Day #6: The Other Web Log

We all know that Apache web server has its access_log log files while IIS has its w3c logs. When people think about web log analysis, they think about the above and only about the above logs, often calling them "the web server logs."  However, the other web log - error_log  (in case of Apache) - contains a lot of fun and useful info as well.

Today's tip is to encourage the review and analysis of this treasure trough - pardon the idiom - of insight.

So, what goes into the error_log? Well, errors, duh! Why errors matter? 'Cause errors often present the only indication that a) you are being 0wned or, worse, that b) you've been 0wned by the attackers.

Additionally, apart from the obvious security usage mentioned above, errors matter since they - or rather, some of them - require actions by the user and thus knowing about them is of huge importance. What is even more striking is that many error messages present an interesting tradeoff - do only a little now to correct the reasons (or, "the root cause" as network management people would say) for the error, or do a whole lot later when the system crashes, gets hacked or "misbehaves" in some other truly spectacular way. Now, a grizzled BOFH would surely say "heh, but who would want to do that! surely, dealing with a dramatic world-ending catastrophe later is more fun that making a minor config fix now."

Now, back to the Apache errors; we are going to show a few examples from a live phishing site, run by the attackers on a compromised server.  We will start from the obvious - server restart, which actually happened as a result of an attack, in our case.

[Sun Mar 12 04:02:05 2006] [notice] SIGHUP received. Attempting to restart

Did YOU restart the server? No? Two choices then - someone else did (likely without permission!) or the server got restarted automatically for whatever strange reason. You certainly need to be at least somewhat concerned in both cases - thus: look for the above messages.

Here are a few more fun examples - the relevance of those for your business is left as an exercise for the readers, but all of the messages below:

[Mon Mar 13 14:54:11 2006] [error] [client 10.10.10.10] Premature end of script headers: index.htm

Does "index.htm" sound like a script to you? It sure should not; and this error message indicates that somebody is trying to run it as a script - a clear indication of malicious behavior. The next message is also similar in this regard:

[Mon Mar 13 14:54:26 2006] [error] [client 10.10.10.10] attempt to invoke directory as script: /var/www/cgi-bin/tcpsupport/

and so is this one:

[Mon Mar 13 14:54:23 2006] [error] [client 10.10.10.10] (8)Exec format error: exec of '/var/www/cgi-bin/tcpsupport/main.htm' failed

Other interesting things spotted on the same server -this one looks like an overflow attack attempt (and, no, the NIDS did not make a squeak about it)

[Thu Mar 19 07:16:11 2006] [error] [client 10.10.10.10] request failed: URI too long (longer than 8190)

So, to conclude, the tips is: when doing web server log analysis, make sure you look at the error logs as well as the access logs.

- Anton

Posted by on December 15, 2006 in LogEd , LogMatters , Security | Permalink | Comments (0)

Visa PCI Fines On The Rise...

From Visa... Fines for PCI Compliance and Data Storage Visa's PCI CAP will build on the company's current enforcement efforts, which include acquirer fines for data compromises involving merchants of any size. Fines are also assessed on acquirers that have failed to confirm that full track data is not retained or that did not provide a PCI compliance plan for their Level 1 merchants by September 30, 2006. In 2006, Visa levied $4.6 million in fines, up  from a 2005 total of $3.4 million.

This new program sets an enforcement date for acquirers to validate PCI compliance for Level 1 and Level 2 merchants. Additionally, Visa is adding new fines to acquirers whose Level 2 merchant customers retain full-track data, CVV2 or PIN data after the transaction authorization.

Specifically for PCI compliance, acquirers will be fined between $5,000 and $25,000 a month for each of its Level 1 and 2 merchants who have not validated by September 30, 2007 and December 31, 2007 respectively. For prohibited data storage, acquirers failing to provide confirmation that their Level 1 and 2 merchants are not storing full track data, CVV2 or PIN data by March 31, 2007 will be eligible for fines up to $10,000 a month per merchant, subject to escalation in the event material progress toward compliance is not made in a timely manner.

Posted by on December 14, 2006 in LogMatters | Permalink | Comments (0)

Descending Into Log Policy Hell

Different countries have very different approaches to Log Management. Across Europe policies vary widely as some specify log destruction within short timeframes while others mandate long-term storage. These same pieces of country legislation often conflict with global industry mandates such as PCI and SOX.

Anton points to the latest out of Germany where "The highest appeal court in Germany has decided that T-Online, one of the largest German ISPs has to delete all IP logs to guarantee the privacy of their customers."

"The decision (German) does not mean that T-Online is now obliged to delete all their IP-logs, the customers first need to complain. "

Anton highlights how bizarre this is: "So, lemme understand, the logic is "surf to an illegal website -> complain to an ISP -> have the proof of that removed."

The decision ends with: "After the district court and the regional court, now the federal appeal court decided that T-Online has no right to store the IP-logs without a legal reason."

Enterprises are increasingly subject to multiple compliance mandates and industry regulations. Not all are in sync and as this highlights, not all are particularly logical. What is certain is that navigating log policy is going to increasingly become a nightmare for global enterprises without automated log management and intelligence.

Posted by on December 05, 2006 in LogMatters | Permalink | Comments (0)

One Of The 10 Most Overlooked Aspects of Security - Log Management

Or so Dark Reading says:

"The trick is learning how to analyze log files in a way that is thorough, yet not too time-consuming. For most IT organizations, this means using a combination of automated log file analyzers, security information management tools, and good old-fashioned detective work."

We agree. Their emphasis on a combination of tools is critical. Anton is quoted in the story:

"To fully realize the value of log data, one has to take it to the next level of log mining: actually discovering things of interest in log files without having any preconceived notion of ‘what we need to find,’” Chuvakin says. “It sounds obvious -- how can we be sure that we know of all the possible malicious behavior in advance -- but it is disregarded so often.”

Posted by on November 30, 2006 in LogMatters | Permalink | Comments (0)

Information Security Must Evolve

Speaking to the need for Information Security to evolve, Amrit makes three key points:

1. The threat environment has become increasingly dangerous.
2. Business is leveraging the Internet for innovation, moving away from brochure-ware to service delivery via the web. SaaS, SOA, web services, are creating complex and dynamic environments in which traditional methods of security and optimization no longer provide the same value to the organization.
3. Regulatory compliance pressures are forcing organizations to gain greater visibility into their security programs.

He goes on to deliver one of the key answers to responding to these challenges -"Process is as important, actually even more so, than technology -start with process than add technology to support strong process, not the other way around".

This is more than right. It's critical to success. Most security teams we speak to are dealing with three levels of "compliance". First, regulatory (SOX, HIPAA, GLBA...). Second, industry and business (PCI...) and third, process and control. Executing against each of these individually would require an unbelievable and unsustainable effort. The reality is that compliance can be addressed best by starting with processes. It should be a "write once, run everywhere" activity.

This is the primary reason we're not just addressing individual mandates through our LogLogic Compliance & Control suites but also best practices and controls such as COBIT, ITIL and ISO. And, why our platform is a SOA that facilitates sharing of information and intelligence with other applications and systems. In doing so, we're getting directly at Amrit's final point: "Security can no longer exist in a silo or a vacuum, security programs and security professionals must align themselves with the business or face extinction."

Posted by on November 13, 2006 in LogMatters | Permalink | Comments (0)

New Jobs This Week @ LogLogic

We're growing our test automation team. Take a look if you are interested in joining one of the hottest start-ups in Silicon Valley.

Posted by on November 03, 2006 in LogMatters | Permalink | Comments (0)

The SOX Burden

U.S. Treasury Secretary Henry Paulson said he is considering recommending changes to the 2002 Sarbanes-Oxley corporate governance law as its restrictions have overwhelmed some American companies.

While the "net result" of stricter reporting standards for executives has been positive, Sarbanes-Oxley has also contributed to "an atmosphere that has made it more burdensome for companies to operate," Paulson said in an interview today from Washington.

"We're going to need to look at how we can address some of these issues," Paulson said. "This is something we're giving a lot of thought to."

Posted by on October 26, 2006 in LogMatters | Permalink | Comments (0)

Reads & Feeds

:: The Compliance Game: "CIOs are still struggling to comply with HIPAA's 10-year-old medical privacy regulations. And the smaller the healthcare organization, the harder the task. Fewer hospitals and healthcare facilities are fully complying with the law this year than in 2005, according to a recent survey by the American Health Information Management Association (AHIMA), a professional organization for health information executives. And more than one-quarter of U.S. security executives whose organizations need to be HIPAA-compliant admit that they are not, according to "The Global State of Information Security 2006," a study released last month by CIO and PricewaterhouseCoopers." More in this report on the state of HIPAA compliance.

:: The Skiny On ITIL: "CIO (a sister publication to CSO) reports that ITIL is gaining steam in the United States and that ITIL "helps IT departments improve their quality of service, including increased system uptime, faster problem resolution and better security." Partly fueled by a tougher regulatory framework—including Sarbanes-Oxley and the Federal Information Security Management Act of 2002—IT vendors and service providers report they are now fielding more requests for information about their ITIL capability."

Posted by on October 25, 2006 in LogMatters | Permalink | Comments (0)

NIST Finalizes Log Management Guide

NIST has published final version of the "Guide to Computer Security Log Management." LogLogic's security veteren Anton Chuvakin participated in the review process. One area that Anton hoped would open up a bit was to eliminate the wall between security uses of logs and other IT uses for troubleshooting and other management issues related to IT and logs.

Customers are increasingly are coming to LogLogic for log management needs far beyond security. Our customers see LMI as a complement to good security. Recently the SANS Institute featured one of our customers in a WhatWorks webinar. He said...

"Initially we thought that log management was going to be part of security monitoring. But then as we started looking at different pieces of the framework, whether it related to security monitoring, maybe our mitigation process, or vulnerability management process, we found that we had all this log information that resides in individual services or platforms around the world, supported by different operations teams.

And to be able to access that data quickly and efficiently, and have it centrally available was key for us to successfully meet our goals for deploying this framework."

The complete archive of the webcast is here.

Technorati : LMI, Log Management, LogLogic, Logs

Posted by on October 17, 2006 in Log Management & Intelligence , LogMatters | Permalink | Comments (0)

Making Sense Of Your Log Files

If you are looking for an independent view of LogLogic appliances take a look at Techworld.

"LogLogic’s products are a bit of an eye-opener, because they make you realise just how relevant your log files are even to non-techies, not least in relation to compliance with modern-day information storage requirements such as Sarbanes Oxley or Basel II. The architecture is cleverly designed, you can deploy LX and ST units in whatever combinations fit your purposes, and they’re easy to use, manage and integrate into your organisation’s network. "

"...you will at least see a huge reduction in the time and effort (and thus cost) of analysing your systems and/or dredging up historical log data."

Posted by on October 16, 2006 in LogMatters | Permalink | Comments (0)

Congress Takes Aim At Log Data

Retaining log data is one area that is getting significant attention as of late as cybercrime continues to be a key focal point for law enforcement authorities. Colorado Congresswoman Diana DeGette believes that no clear guidelines are in place to either protect privacy or secure onine users, and late last week said she will be introducing legislation to force Internet providers to retain customer data for at least one year.

Plenty of recommendations are out there - for instance, Basel requires data retention from 3-7 years, PCI recommends a minimum of one year of data retention. And equally important is how the data is stored and managed. For example, is the data secure in transport and at rest? Is it an immutable set of data or processed data and how do you handle disposal?

What would legislation look like? DeGette suggests that:

One form could require Internet providers and perhaps social networking sites and search engines to record for a year or two which IP address is used by which user. The other form would be far broader, requiring companies to record data such as the identities of e-mail correspondents, logs of who sent and received instant messages (but not the content of those communications), and the addresses of Web pages visited.

DeGette said Thursday that her proposal would not require retention of the communications themselves, but would identify data so that law enforcement officials--if they had probable cause to believe a crime has been committed--could go in and get a subpoena and subpoena these IP addresses.

Regardless of how legislation aims to deal with the issue of cybercrime, one trend that we are seeing at LogLogic is individual sites proactively alerting patrons of their policies regarding log data. PopSugar, a red-hot celebrity gossip site based in San Francisco, has a posted privacy policy that calls out log files in great detail. PopSugar alerts patrons of their site that they use log data "to analyze trends, administer the site, track user's movement in the aggregate, and gather broad demographic information for aggregate use. These log files are not linked to personally identifiable information. We may use a tracking utility that uses log files to analyze user movement."

Whether it is for legal reasons, to track down the bad guy, or just to gather marketing data on viewers, log data a systematic and managed approach to collecting log data is critical.

Technorati : Compliance, Logs, PCI, PCI Compliance

Posted by on September 25, 2006 in Compliance , LogMatters | Permalink | Comments (0)

Big Fish Swallows Small Fish

EMC today swallowed the #3 player in security event and information management – adding to the security acquisitions of recent months.  With newly acquired with Windows-based SIEM capabilities, they enter a market rich in players like CA, IBM, Cisco, Symantec and ArcSight.

Today’s announcement validates what we have been saying all along – SIEM is an application and feature within a broader security framework. And, LMI is a separate, larger market and opportunity – SANS estimates G2000 Enterprises spend at upwards of $1bn USD per annum. SIEM vendors are not visible in the majority of LMI deals underscoring that customers are looking at LMI very differently than they look at SIEM.

As seen with this announcement, SIEM is being absorbed as a security "feature" into identity management (Novell, IBM), threat mitigation (Cisco) and now storage management (EMC) whereas LMI is continuing to grow in importance as independent market (SANS) driven by regulatory compliance and IT operations and service management. Features like Open Log Services and Universal Log Processing will fuel the compliance, operations and security offerings, driving new services and integration opportunities. And, vendor neutrality coupled with partnerships with the likes of IBM, NetApp and CSC puts us in a great position to best serve customer needs.

The acquisition further emphasizes the need for open log management and intelligence (LMI) to fuel a broad range of applications while answering compliance and IT control mandates – something none of the big guns have today. In a market where LogLogic is now the undisputed leader still no player other than LogLogic has a major capability or competency in LMI and we will  continue to extend our market share leadership position where customers will benefit from our continued innovation and focus.

All in all a great announcement for EMC and for us – validates the market for LMI and SIEM as a feature of broad security offerings; grows the momentum behind use cases for log data; and strengthens our position as the vendor neutral, undisputed market leader in LMI.

Posted by on September 18, 2006 in LogMatters | Permalink | Comments (0)

Survey: Most insider-related data breaches go unreported

A survey today from the Ponemon Institute says that most insider-related data breaches goes unreported:

"We found that many of the respondents in our study found that it was difficult, if not impossible, to identify all data breaches that exist -- and over 79% of the respondents said one, if not more, insider-related security breaches at their companies go unreported," said Larry Ponemon, chairman of Ponemon Institute. "Because it's insider-related normally, involving a careless or negligent employee [and] not an evil employee, maybe they are more likely to go unreported because people know each other, and maybe because people know each other, they say it was a mistake and maybe in the future they'll fix it."

This really makes the case for automation and real-time reporting and analytics on IT controls. The survey goes on to flag some other interesting points, notably that:

The respondents said they devote a considerable amount of their efforts to trying to prevent or control insider threats as part of their company's IT security risk management program. Approximately 10% said they spend more than half of their time on insider-related risks, and about 55% of respondents said they spend more than 30% of their time dealing with those issues, according to the survey.

Next generation Log Management and Intelligence solutions specifically reduce the human resource requirement in protecting information assets. According to The Ponemon Institute, "...the National Survey on Managing the Insider Threats calculated the average annual cost of insider data breaches at $3.4 million, and found that spending on technologies and programs aimed at addressing the insider threat seemed insufficient."

Source: Survey: Most insider-related data breaches go unreported

Posted by on September 12, 2006 in LogMatters , Security | Permalink | Comments (0)

Join LogLogic

We're hiring! If you are interested in joining a red hot Silicon Valley-based enterprise that is the leader in its category, drop us a line. Especially take a look at the just posted role in sales operations. The benefits and people are great!

Technorati : Jobs, LogLogic

Posted by on August 23, 2006 in LogMatters | Permalink | Comments (0)

The Case for a Vendor-Neutral Open Log Standard

Using standards for defining the log output from common types of devices and applications is a good way to improve interoperability, eliminate vendor lock-in and generally improve business as a whole. That is, if the standards are created through an open, collaborative effort.

Over the past few months we have discussed this very challenge here at LogLogic with our customers, our partners and external parties such as the SANS Institute, NIST, Gartner, Mitre and others with the aim of thoroughly vetting what an Open Log Standard and Initiative would look like. It is a conversation many have been leading.

Mary-Ann Davidson, CISO of Oracle, has been promoting an audit log standard for years. Others include a spring initiative by NIST to launch Common Logging Interchange Format. SANS deserves credit for picking up the ball where NIST left off. They brought together a wide range of users and "loggies" to debate standards at the recent log management summit. And, Amrit Williams from Gartner also published on the topic - such as his May 2006 Gartner publication #G00139205 on log output standards.

With so much interest, there are inevitable proprietary vendor announcements that are - like most things with the vendor label - closed in nature. Initiatives such as these typically fail for a simple reason - they depend on a company rather than the community to succeed. The last thing coders, technology inventors or enterprises need is a vendor specific common event format. What in fact is created is "uncommon event formats". These "uncommon event formats" only bring another layer of complexity to an already complex problem by driving the customer to adopt a vendor centric, rather than neutral, solution. Take IBM's uncommon "Common Base Event" - also for logging, tracing, management and business events. At least IBM claims theirs is an implementation of the OASIS "WSDM Event Format".

There is a bigger and more important point though that is missing from conversations related to log standards. That is, any conversation related to standards for log output should start with a discussion about the use cases for log data - a discussion about the best practices of using information contained in logs for operational excellence, IT control and compliance. That is a customer discussion about best practices and use cases - not a vendor discussion. A standard should be defined top-down with the customer in mind - perhaps by using frameworks such as ISO 17799, COBIT and ITIL as a starting point to deduce logging requirements - rather than bottom-up, using a random vendor's architecture for security event reduction (note: not an architecture envisioned for operational excellence, IT control or compliance in the first place ...) as an unnatural starting point.

At LogLogic, we envision a broad initiative to create a Open Log Community with participation in defining key standards, best practices and techniques that benefit all stakeholders. Tackling the log standard conundrum from the perspective of the broader community will benefit our common key constituent -- our customers.

Technorati : Log Management, Log Management & Intelligence, LogLogic

Posted by on August 22, 2006 in LogMatters | Permalink | Comments (0)

Relationships Of Necessity

Optimize has an interesting piece on how trends such as increasing regulation is altering the relationship between CIOs and CFOs. They argue that "in effect, the door between the CIO and CFO offices is opening wider, and the executives are building a path to regular interaction that allows the IT function to excel in meeting the business' needs, not simply to comply with regulatory mandates or budgetary strictures."

We're seeing the same - what starts as a project to satisfy regulations or mitigate risk ends with a broader business improvement mandate. They make the case: "Several factors have an immediate impact on the level of collaboration between finance and IT. First, increased scrutiny of financial reporting has contributed to a significant number of earnings restatements in the past year. In the United States, we saw 414 restatements in 2004, compared with 270 in 2001, according to Huron Consulting Group's latest "2004 Annual Review of Financial Reporting Matters." That fact underscores the need to design and coordinate information and technology systems across the enterprise-something to which CIOs have been calling attention for years and which CFOs are now demanding."

Technorati : Compliance, LMi, Log Management

Posted by on July 20, 2006 in LogMatters | Permalink | Comments (0)

PCI Updates Coming

Visa, MasterCard plan to unveil new security rules in the next couple of months. According to Computworld.

The rules will be the first major updates to the one-year-old Payment Card Industry (PCI) data security standard, which analysts said is slowly but surely being adopted.

One set of PCI extensions is aimed at protecting credit card data from emerging Web application security threats, said Eduardo Perez, vice president of corporate risk and compliance at Foster City, Calif.-based Visa. Other new rules will require companies to ensure that any third parties that they deal with, such as hosting providers, have proper controls for securing credit card data....

...The number of companies complying with PCI requirements finally appears to be picking up after a slow start, several analysts said. Visa says that about 22% of Tier 1 merchants, which the company defines as those processing more than 6 million card transactions per month, are already PCI-compliant, with another 72% on track to becoming fully compliant.

Posted by on July 20, 2006 in LogMatters | Permalink | Comments (0)

Don't Ever Find Yourself In This Position!

Next generation log management systems will enable you to avoid many of the issues outlined in this recent article in InformationWeek.

First, in creating immutable logs ensure that you keep a secure back-up that is tamperproof. You'll avoid responses like this: "''I couldn't look at all the data; said Faulkner, when defense attorney Chris Adams questioned him about having backup tapes instead of forensic mirror images to analyze in the case. ''They were just the active data. When I ran it, it asked for Tape 2 but there was no Tape 2 The information for the [central server] wasn't a forensic image. To preserve digital evidence, a forensic image is best practice.''

Creating a forensic image of log data is critical. Erin Kenneally said it well in FSA Times:

"With forensically sounds logs, companies can reduce the potential of loosing a lawsuit, diminish the costs associated with discovery and defense, increase the likelihood of forcing an opponent into settlement, and be a resource to define against actions related to corporate governance."

Second, log everything. Distributed log management systems enable 100% logging, and next generation log management solutions provide you with assurance that no log was left behind.

Third, your log management system should provide complete chain of custody over your log data. You will know who accessed what, when and what they did. Your log data can be trusted as a form of evidence because with a next generation log management solution as you will know if they were edited by a root user.

Whether prosecuting or defending cases, or just executing best practices, immutable logs can play a critical role in attesting to controls, compliance and the quality of evidence. Here is another great quote:

"When audited logs are immutable and cannot be altered, there are additional advantages for deterrence and proof of policy or legal violations With immutability, deterrence may be improved for all users of the system." Marble Foundation. Implementing a Trusted Information Sharing Environment. February, 2006

Posted by on July 18, 2006 in LogMatters | Permalink | Comments (0)

Five To One...

The vendor panels at conferences are usually quite entertaining and yesterday's at SANS was no exception. Someone posed a question for Anton on how many appliances would it take to handle 100,000 messages per second. Anton responded correctly with two - two LogLogic ST Appliances.

Alan (moderating) then went on to ask Network Intelligence the same question. Well, they said, they have one appliance that can handle five thousand messages and another that can handle ten thousand. And, then, they said LogLogic couldn't handle 50,000 messages. Now, normally we'd duke it out on the stage with a little "I can, you can't... Can so, can not...".

But then a little yellow note appeared from the audience - Alan read it and it went something like this: "We've tested LogLogic in our Labs, they can handle 50,000 messages per second". So, what could be better than us saying it than an end-user!

So in answering the original question: In this scenario, for every LogLogic system you would need five times more NI appliances. That's two LogLogic ST appliances at 50,000 messages per second, or, ten Network Intelligence systems running at 10,000 messages per second. I know what solution I'm picking...

And if you are in any doubt, here are testing results from SANs on our LX2000 appliances. Note that unlike many players in the market, our specified loads are not peak loads - so the systems will deliver even more "bang for the buck" than stated.

Couple of other observations from the day... It is clear that most enterprises are using a mix of solutions. This places an increasing priority on selecting vendors who can deliver services oriented architecture and Open Log Services. You want to be able to share data between your SIEM and your Log Management Solution. And, the time is right of log standards - this needs to be a community effort.

Technorati : LMI, Log Management, LogLogic

Posted by on July 13, 2006 in LogMatters | Permalink | Comments (0)

At the SANS Log Management Summit

Anton, Jill and myself are here in DC at the inaugural SANS Log Management Summit. Alan Paller opened the morning session to a packed room - over 200 people are attending - stating that Log Management is the sweet spot that bridges compliance and security.

Ben Wright took the stage after him speaking on Logs & The Law. A couple of the highlights from my rough notes:

1. Have a written policy... it is better to speak in general terms rather than hard language. And, a written policy, not followed = a negligence case.
2. Keep logs of log management - records about what was reviewed. Ben saw lots of value in this saying that "logs of log management are more valuable than logs themselves".
3. Bias to keeping more logs - general trend in law which rewards organizations that keep more information than they did in the past. More expectations for records/archives.
4. Only full audit committee should have power to know all logs... Maintaining custody of your log data is something we constantly emphasize and is a key feature of LogLogic.

Watch for our SANS Webinar on Logs & The Law - with Ben - later this month.

A great customer panel followed featuring a very large financial institution - and LogLogic customer, NetApp and others. NetApp are collecting some 40 million log messages a day - and emphasized that you can't use people exclusively to do log data mining when processing these kinds of volumes. You need a great tool. They made some other great points that we often look at when architecting solutions. You can't just collect logs from one set of devices and they shouldn't be dispersed. Get them centralized for rapid review and forensics. There was a definite preference on the panel for agentless log management systems.

- Andy

Posted by on July 12, 2006 in Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

SANS Log Management Summit

Anton, Andy and Jill are off to SANS' Log Management Summit. The fact that SANS can pull a hundred-plus crowd along with worldclass speakers is further evidence of the strength of the log management and intelligence market. Watch for our announcements on Thursday for further proof. If you are at the summit and would like to connect, look out for the LogLogic logos or drop us an email.

Technorati : LMI, Log Management

Posted by on July 10, 2006 in LogMatters | Permalink | Comments (0)

SOX Audit Fees Rising...

SOX Audit fees keep rising according to Inside Sarbanes-Oxley and the Chicago Business Journal:

Since federal accounting reforms were enacted in 2002, public companies with less than $1 billion in annual revenue have seen audit fees nearly triple in the past four years, according to a recent study by law firm Foley & Lardner. Audit fees were more than $1.2 million for fiscal year 2005 compared to $332,000 before accounting reforms.

We've established with several customers that using the LogLogic Compliance Suite - SOX Edition can significantly reduce secondary audit requirements and overall audit costs. Drop us a line to learn more.

Technorati : Compliance, SOX, Sarbanes-Oxley
Del.icio.us : LogLogic

Posted by on June 19, 2006 in LogMatters | Permalink | Comments (0)

The Case for Universal Log Processing

Andreas Antonopoulos revealed some more wisdom from his interviews with 82 IT managers:

“The average enterprise has more than 1,200 applications, but even that number is increasing every year”.

I have always known that enterprises have many applications because a typical log management and intelligence RFP has requests for about 200 discrete log sources. The bigger point is that 70% of enterprise software applications are homegrown or highly customized. This means there are a lot of “one-off requests”.

Therefore, it doesn’t matter anymore how long your list is of “supported applications”. What really matters is how quickly you can support “unknown log types”. Nothing is better than instant gratification and so at LogLogic have opted for out-of-the-box support for unknown log types. We use natural language processing and machine learning algorithms to provide baseline support for any application that writes ASCII.

- Dominique

Posted by on May 31, 2006 in Log Management & Intelligence , LogEd , LogMatters | Permalink | Comments (0)

Vote For LogLogic

We're up for another award, this time over at Info Security. If you are a customer, you can vote today!

Posted by on May 16, 2006 in LogMatters | Permalink | Comments (0)

SANS Log Management Summit

In what is clear recognition of the distinct Log Management and Intelligence (LMI) Market, SANS is staging what is to the best of our knowledge, the first LMI Summit.

The SANS 2006 Log Management Summit, on July 12-14 in Washington DC at the Wardman Park Marriott Hotel, is a must-attend event that focuses on what works well in log management – the best practices. 

The Log Management Summit is a user-to-user, non-commercial conference where you can learn about the strengths and weaknesses of competing technologies, where users will share the lessons they have learned and take an early look at the results of SANS’ assessment of what types of log management intelligence and reporting are most effective in actually improving security. The Summit is your opportunity to go beyond regulatory demands, gain control of your log management and ensure that the system your organization has in place is doing all that it can to improve your organization’s security.

Posted by on May 08, 2006 in Log Management & Intelligence , LogLogic News , LogMatters | Permalink | Comments (0)

Network Management Innovation

Computerworld covers the increasing investment in LMI - including in LogLogic. James Govenor of Redmonk is quoted throughout the piece:

"Network management tends to be real time; log management is after the fact -- it's more about looking at what happened and analyzing that," Governor said. "These companies are making log management more of a real-time function, and then it becomes more valuable. It's moving from being a subset of security management to more of an application-management function."

What James is getting at here is important. First generation solutions didn't collect or index in real-time. They depended on costly and time consuming rules writing. They correlated security events. As a result they were and are principally about what happened.

Next generation solutions such as those from LogLogic are real-time, they replace rules with Open Log Routing (enabling rules to be aggregated with an MSSP or application) - and machine learning. The correlate across all IT infrastructure. As a result they are as much forward looking as about what happened. They enable you to anticipate and predict risks.

At the same time, LMI solutions are moving well beyond security. We see more application today in the areas of IT control and process automation for compliance. And other areas such as application monitoring - for instance, areas of the supply chain being used more and likely to require additional capacity or management.

Posted by on April 11, 2006 in LogMatters | Permalink | Comments (0)

What To Look For In A Service-level Agreement

In recent years, security outsourcing has become a popular and viable means of lowering the cost of perimeter security management - Jian shares his thoughts in two pieces in Computerworld - Part 1 & Part 2.

Anyone thinking about outsourcing such a mission-critical aspect of their network should understand in detail the potential implications to their IT security infrastructure and their company as a whole. One of the biggest differences among providers of security services is the service-level agreement (SLA). In this five-part series of articles, we will dive deep into the various aspects of the SLA and attempt to explain in details what the SLA should contain and why each of the items is necessary.

Working with partners such as Counterpane, LogLogic is ushering in a new generation of managed log services which take advantage of out Open Log Routing platform and machine learning technologies. You can register and listen to our last webcast together or give us a call for more info.

Posted by on April 11, 2006 in LogMatters | Permalink | Comments (0)

RedHerring 100 Finalist & New Sales VP

RedHerring announced their list of finalists for the "Red Herring 100 North America" and LogLogic has made the cut. They say that "well over 1,000 privately held technology companies submitted to this year's edition of the prestigious award, giving evidence to the invigorated innovative and entrepreneurial strength of the technology ecosystem."

On top of our NetApp news earlier in the week we also welcomed Robert Yusin to LogLogic's executive management team - Robert is responsible for all things sales in Nth America and Europe. Robert joins us from Symantec most recently, and prior to that Finjan Software and Commerce One.

On NetApp - James had  this to say on our partnership while media are also picking-up on the story.

Oracle CSO Mary-Ann Davidson has an interesting piece on creating a log audit standard:

...the government is in the business of promoting the public good. It’s one of the reasons it exists. Having a common logging and auditing standard promotes the public good.

To make an off-line analogy, how well would 911 work if, in each jurisdiction, the phone number to call for assistance were different? How well would it work if a user in Omaha called up 911 and said, “He pilikia ko'u!”? (That's Hawaiian for “I have a problem!”) Without a common framework and language for auditing and logging, we can’t begin to address problems in real time.

Posted by on April 05, 2006 in LogMatters | Permalink | Comments (0)

What LogLogic Really Does

Redmonk's post last week on the Log Management & Intelligence (LMI) market has been drawing quite a bit of attention. Some of it on the mark, some of it not.

SecurityIncite is the latest to weigh in, agreeing that this in a seperate market. Where this post misses the mark is understanding what LogLogic does. We do collect Log data. More logs, from more sources, at faster rates than anyone in the industry. We don't stop there.

Using machine learning and behavioral anomly detection we then provide real-time alerting and reporting. This moves LMI from reactive to proactive mode. By detecting subtle shifts in IT behavior and trends we enable customers to prevent harmful activity and mitigate risk before it becomes business impacting. So, we are analyzing it in real-time.

We also provide correlation. This is often a misunderstood offering from LMI vendors. LogLogic provides root-cause correlation of log data from any device type and vendor by IP address, user name, system name, geography, time and such. Correlation is available for alerts, search, ad-hoc and scheduled reports.

Unlike with most other solutions, correlation for alerts and search can be achieved for pre-defined strings and for any keyword, Boolean combination of keywords or complex regular expressions.

• An example of a correlated alert: “if x regular expression occurs more than three times per minute across the IT infrastructure including servers, routers and applications, then trigger an alert via e-mail or SNMP”.
• An example of a correlated search: “look for y regular expression or z user name across the IT infrastructure including servers, network devices and applications from time period a to b in the past”.

LogLogic is unique in then providing a platform for unchanged and tamper-proof storage of log data. Up to 24 terabytes on a single appliance. We also provide the engine that determines who gets access to what.

Then, through easy-to-use templates, you can assemble more than 13,000 different reports - have them generated and distributed automatically - or, make use of our Complaince Suite that provides more than 175 reports and alerts on COBIT 4.0 processes.

Not all vendors go beyond providing a "rear view mirror" into LMI. LogLogic does - and provides much more.

Posted by on March 27, 2006 in LogMatters | Permalink

"PodSlurping"

Now here's a new phrase - Podslurping:

"Nobody wakes up in the morning worrying about antivirus or their firewall because we all know we need those things, and we all have them in place," Burton said. "Now the greatest threat is very much inside the organization, but I'm not sure there are that many businesses (that) have realized it's possible to plug in an iPod and just walk away with the whole business in a matter of minutes." - Abe Usher

All the more reason to have an effective log management and intelligence solution in place with beahvioral anomoly detection giving you real time alerts when data shifts in abnormal ways.

Posted by on February 15, 2006 in LogMatters | Permalink | Comments (0)

LogLogic Career Fair

Not your everyday blog entry but if you are into logs and looking for a new gig with a hot company we definitely want to meet you.

Attend LogLogic’s Career Fair Wednesday, January 11th 4pm-8pm

LogLogic Headquarters
3061-B Zanker Road, San Jose, CA 95134

Bring your resume or paste your resume into an email with “POSITION – YOUR NAME” in the subject line to: careers@loglogic.com, Attn: V. Golub. For more information on the jobs we have open, visit LogLogic.com.

Posted by on January 05, 2006 in LogMatters | Permalink | Comments (0)

SANS To Start College for IT Security...

Congrats to the team at SANS in getting approval by the State of Maryland to create a college focused on security. SANS says they plan to begin accepting candidates for Masters of Science degrees in information security. SANS will offer MS degrees in two subjects: Information Security Engineering and Information Security Management.

Posted by on December 22, 2005 in LogMatters | Permalink | Comments (0)

Log Data Management: A Smarter Approach to Managing Risk

Dominique's recently published article is ready for download:

To manage risk intelligently and cost-effectively, companies are looking to their network log data for answers. Log data provides a complete audit trail of user and system activity, while delivering critical decision support to mitigate security and performance incidents. Fortunately, log data is readily available in any data center, and companies need only find effective ways of collecting, aggregating and archiving it to put it to use.

Posted by on December 21, 2005 in LogMatters | Permalink | Comments (0)

Logs Need Attention Too!

Douglas Schweitzer flags Jian's recent article in ComputerWorld and says:

I’ve said it before and I’ll say it again: remember that valuable nuggets of information reside in your logs. If you take the time to learn log analysis, your firewall and IDS reports will fulfill their raison d’etre revealing much valuable data

.

Posted by on November 18, 2005 in LogMatters | Permalink | Comments (0)

Welcome...

Welcome to the LogBlog, LogLogic's Blog on all things to do with us and, data log management and intelligence. We're all about the logs that come from IT. Think storage, applications, networking, servers and operating systems. Don't think logs from trees.

Our basic categories work like this:

• Blinks: short snippets of news and other stuff we're pointing to that could be of interest
• Innovation: thinking and observations from our scientists and geeks
• Log Management & Intelligence: General thinking and observations on log management and intelligence best practices
• LogEd: Updates and content from our education program
• LogLogic News: breaking company news
• LogMatters: highlights from our eZine and newsletters
• Risk Management: thinking and observations on how log management and intelligence can be applied to reduce enterprise risk
• Security: same as above but for security
• Compliance: same as above but for compliance

You will see posts from Andy Lark, CMO; Dominique Levin, VP product marketing; Jian Zhen; and many more - we hope. We're going to keep the content mixed - technical and business. This reflects what our customers asked to hear. Give us feedback and share thoughts.

Posted by on September 20, 2005 in LogMatters | Permalink | Comments (0)

Visit loglogic.com

Subscribe to this blog’s feed

• June 2010
• May 2010
• April 2010
• March 2010
• February 2010
• January 2010
• December 2009
• November 2009
• October 2009
• September 2009
• August 2009
• July 2009
• June 2009
• May 2009
• April 2009
• March 2009
• February 2009
• January 2009
• December 2008
• November 2008
• September 2008
• August 2008
• July 2008
• June 2008
• May 2008
• April 2008
• March 2008
• February 2008
• January 2008
• December 2007
• November 2007
• October 2007
• September 2007
• August 2007
• July 2007
• June 2007
• May 2007
• April 2007
• March 2007
• February 2007
• January 2007
• December 2006
• November 2006
• October 2006
• September 2006
• August 2006
• July 2006
• June 2006
• May 2006
• April 2006
• March 2006
• February 2006
• January 2006
• December 2005
• November 2005
• October 2005
• September 2005

Blogroll

• Whatever Compliance
• CynergisTek
• Bruce Schneier
• InfoSecDaily

Compliance

• Compliance Pipeline
• SOX Compliance Journal
• Sarbanes-Oxley 101

Good Reading

• Internet and Web Security Review
• Log Management ROI
• Log Management Best Practices
• SANS Institute Log Management White Paper

LogLogic

• O’Reilly
• Anton Chuvakin
• LogLogic.Com
• Jian Zhen

LogLogic Partners

• Juniper Networks
• ONStor
• Counterpane
• Bluecoat

Sites We Watch

• Verizon Business Security Blog
• SANS
• IT Manager's Journal
• CIO Insight
• Security Focus
• TechWeb
• Slashdot
• InformationWeek
• The Reg
• C/Net
• OSF DataLossDB | Data Loss News, Statistics, and Research