« Critical Log Management Questions - Answered! | Main | Logging Poll #8 Context for Log Analysis »
The best of the 80s: log management for operations!
Log management has been around for a loooong time. In the 80s log file management was the primary mechanism for fault analysis and management of computer systems. Also in the 80s, Eric Allman at the University of Berkeley developed a logging standard called syslog as part of the Sendmail project. While adopted by quite a few applications, many other protocols and formats persist until today.
The sheer success of log data nearly killed it. The cacophony of log formats and the sheer volume of messages generated – up to 40 terabytes a month for a mid-sized organization or, shall we say 100,000 log messages every second (!) , it is impossible for any human being to keep track of all that logs have to say. Based on SNMP alerts and other event data, including selected error log messages, large-scale event management systems such as HP OpenView emerged as the new kings of fault detection.
If it was not for compliance and security concerns, log management might not have made it back. But out of a need to track user activity and to identity potential insider and outsider intrusions and transgressions of corporate networks emerged a new form of log file analysis. Log data featured prominently in Paul Proctor’s Practical Intrusion Detection Handbook in the late 90s for example and tens of companies emerged to perfect the art of security event management based on log data.
Now, in part due to virtualization and the ever increasing cost of downtime in our networked economy, system and network administrators have re-discovered log data. In surveys, 70%+ of organizations confess their primary budget for log management still comes from compliance. However, this same group admits for years now that 70% of their use of log data is driven by operational needs such as fault detection and problem isolation. This is no surprise, because operations use cases can drive true log management ROI. One minute of down-time could cost millions so if automating log management can help to accelerate problem isolation, then companies are willing to pay big bucks. If giving help-desk employees access to normalized log data can off-load expensive third-level support personnel that is even better.
So, as the sun is setting on HP OpenView (the name was changed to HP Software in 2007), a new dawn has broken for log management in operations! Hoorah!
Posted May 01, 2008 in | Permalink
TrackBack
TrackBack URL for this entry:
http://www.loglogic.com/mt/mt-tb.cgi/320
July 2008
| Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | ||
| 6 | 7 | 8 | 9 | 10 | 11 | 12 |
| 13 | 14 | 15 | 16 | 17 | 18 | 19 |
| 20 | 21 | 22 | 23 | 24 | 25 | 26 |
| 27 | 28 | 29 | 30 | 31 |
Categories
- Blinks
- Cloud Computing
- Compliance
- Innovation
- Log Management & Intelligence
- LogEd
- LogLogic News
- LogMatters
- Project Lasso
- Risk Management
- SaaS
- Security
Archives
- July 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- August 2007
- July 2007
- June 2007
- May 2007
- April 2007
- March 2007
- February 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
- August 2006
- July 2006
- June 2006
- May 2006
- April 2006
- March 2006
- February 2006
- January 2006
- December 2005
- November 2005
- October 2005
- September 2005