"make machine data accessible, usable and valuable to everyone. Machine data is a fast growing and pervasive part of “big data”— generated by every component of IT infrastructures, applications, mobile phone location data, website clickstreams, social data, sensors, RFID and much more."
Modern application management requires speed to support business’s complex and ever-changing application requirements. At the same time, we need to make sure our applications don’t break down. And the stakes are higher than ever—IDC (DevOps and the Cost of Downtime: Fortune 1000 Best Practice Metrics Quantified, Stephen Elliot, IDC Research, December 2014.) reports that the average cost of a critical application failure is $500,000 to $1 million per hour. Worse, business credibility and even business viability are at risk with repeated outages.
Applications Are More Complex Than Ever. Today’s IT is disjointed and more complex. The customer experience is now defined by elements like cloud infrastructure, software, APIs, microservices and network performance. And all of these reside outside the core application. Mobile apps perform a tremendous amount of code execution, making mobile app clients a potential source of failure. Native app code bases also vary by distribution that users may (or may not) adopt, meaning you have to manage and support a large, complex matrix of mobile app versions.
When it comes to addressing issues, APM tools can detect problems with availability and performance, but that’s like only seeing part of the problem. Unless the problem is related to application code, we don’t know what the problem is. That’s why configuration logs, automation tools and configuration state changes often become “the smoking gun” of evidence for where problems may have been introduced.
Combining APM data with other kinds of data becomes critical. We need to be able to rapidly triage, troubleshoot and remediate problems, and that requires other data sources—like log files—that can help identify the specific cause of a problem.
Emerging Application Architectures The rules of the application are changing quickly—as webscale IT becomes a new mandate, new architectures and approaches are required to support it. Moreover, businesses have a mandate to deliver applications quickly and iterate often. To do this, many organizations are turning to an API-led approach to provide scale, agility and flexibility. But their use and performance needs to be monitored and analyzed. Microservices are developed to operate independent of each other and are accessed via a single API. As a result, they increase the number of elements that need to work together to generate a completed transaction. Container-based technologies like Docker provide a host environment that’s perfect for running microservice instances. Containers can be started and stopped on demand, and they can easily move between machines.
This shift means:
• There are more technologies we are responsible for managing (many of which are not our own). That also means there are more points of failure than ever before.
• Taking a data-centric approach to managing services is imperative. Data, not quantity of servers, should define the value you derive from your management approach.
• Focusing exclusively on code performance and app servers doesn’t give you the full picture anymore.
Transcending the Silos
When users see an app, they see our business. But under the hood of any app are silos and tiers— made up of networks, systems, containers and virtual machines, application tiers, APIs, microservices, databases, load balancers, cloud services, firewalls, power, HVAC, and storage—all of which can cause problems.
Just as IT needs a management tool that addresses overall service availability and performance, application managers need a platform that transcends the silos—incorporating and analyzing data from the various sources that influence performance and availability.
To do this, the platform must collect, index, store and analyze data and be able to focus on event sequences or even individual data points. This approach gives you complete visibility into our apps—enabling us to monitor application performance, troubleshoot problems and analyze applications—resulting in improved future releases.
While metrics, logs and data coming in from other tools are all valuable for monitoring, most often, log files become the most authoritative source of data when performing detailed root-cause analysis and troubleshooting.