Logblog: Enable visibility and transparency through application logging

« Log management – a safe bet for 2009? | Main | LogLogic named a finalist for SC Awards 2009! »

Enable visibility and transparency through application logging

LogLogic has been advocating comprehensive logging for all IT components (or configuration items if you are in the ITIL camp) including applications for a long time now. We have worked with many of our customers to ensure that there's 100% collection and analysis of their IT log data. In the last several months there's been a huge uptick in the area of application logging, specifically for the application developers. This is partially due to the general interest in cloud computing and SaaS applications.

To quote a few blogs, Amrit Williams said in his blog "Amazon AWS, Google App Engine, Microsoft Azure, and More - Part 1: Can We Secure The Cloud?" (emphasis mine):

The one suggestion that elicited the greatest interest and most questions was a simple one; develop your applications so that they can be easily audited by the security and IT teams once they are in production, enable auditing that can capture access attempts (successful or not), date/time, source IP address, etc…the folks I talked to afterwards told me it was probably the single most important concept for them during the summit - enable visibility.

Todd Hoff said in "Log Everything All the Time":

you need to log everything all the time so you can solve problems that have already happened across a potentially huge range of servers.

What you need to be able to do is trace though all relevant logs, pull together a time line of all relevant operations, and see what happened. And this is where trace/info etc is useless. You don't need function/method traces. You need a log of all the interesting things that happened in the system.

Todd also gave a fairly extensive list of suggestions to application developers on how they should be logging in his article.

By logging, capturing and analyzing everything, IT organizations can enable visibility and transparency into their applications. This not only helps with troubleshooting and forensics as Todd suggested, but it will help IT organizations achieve and enhance accountability. It will help IT do more with less.

Bottom line:

Log everything all the time
Capture and centralize archive all logs
Analyze (search, report, correlate, alert, trend) all logs to understand what's happening in your IT infrastructure
Continuously monitor the trend (automatically or manually) to ensure no anomalies will occur without notice

Posted December 03, 2008 in Log Management & Intelligence | Permalink