What is Log Analytics and how can it be used as part of my Security Operations?
That is a question that many Security Professionals may be asking (or should be). Log Analytics provides a cloud-hosted log repository and management solution. In AWS terms, Log Analytics is similar to AWS Cloudwatch, with a few slight differences. As more workloads shift to Azure, the need to have clear and accurate logging for security monitoring is critical. In this article, I’m going to explore the basics of Log Analytics, and then Azure Sentinel which is built on top of Log Analytics to provide anomalous activity detection. Using the two solutions together, any organization can begin to create a security analytics program without spending millions on SIEM costs.
Log Analytics History
Log Analytics was born out of a not too widely known service called System Center Advisor (SCA) in 2012. SCA of course was part of the System Center brand of server tools, but SCA was a web service that provided health and tuning guidance. In 2012, Microsoft was all about selling products and tools for the data center, and SCA was part of that line-up.
In 2014, System Center Advisor became Azure Operational Insights. I don’t feel that Azure Operational Insights got much traction, and it really did not get used a lot. This was a time when Microsoft was in the process of shifting its focus to the cloud. By 2016, Azure Operational Insights was rebranded Log Analytics (OMS). OMS is Microsoft Operations Management Suite in Azure. In 2019, Microsoft has somewhat dropped the OMS branding and you just see Log Analytics in the Portal now.
While Log Analytics can capture Windows Event logs via an agent, most of the logging is coming from Azure PaaS and SaaS sources. At this point, most of the major Azure services have an option to use Log Analytics, so this provides a central place to detect anonymously behavior in Azure. In this regard, Log Analytics is similar to Splunk or Exabeam Data Lake.
Log Analytics Pricing
Log Analytics Pricing is passed on two components, Data Ingestion and Data Retention. The first 5GB of Log Ingestion is free, after that its $2.30 per GB per month. The first 31 days of log retention is free, after that, you pay $0.10 per GB per month. However, if you are using Azure Sentinel, you get 90 days of retention at no charge.
In general Log Analytics is fairly affordable, especially if compared to other SaaS solutions like Splunk. So its relatively inexpensive for small organizations to start leveraging. In my POC, which mimics a small business using Office 365 and Azure, the monthly cost was under $10/month.
Enabling
Log Analytics can be enabled across most of the common Azure services, and the list appears to be growing daily. It feels pretty safe that Log Analytics is here to stay. The one downside of Log Analytics is there is no central configuration for logging across a tenant. That means you have to touch the Diagnostics settings in each service or configure via automation.
I’m going to touch on a few of the more common scenarios, Virtual Machines, Azure SQL, and App Services.
Querying Events
Now that we have logging enabled, we can start querying this data and using it for security analytics. Log Analytics uses the Kusto Query Language. Kusto is case sensitive, and keywords are typically written in lower-case.
Queries can be relatively simple. The following query, searches for the first 10 records in the SecurityEvents table.
SecurityEvent | take 10
Wrapping Up
Log Analytics provides a good foundation to start capturing log data from Azure services that can be used for Security detection or forensics. This can be done stand alone by simply using the Log Analytics query interface, using Log Analytics in conjunction with Azure Sentinel, or by feeding Log Analytics data into another SIEM like Splunk or Exabeam. Whichever approach you take, it is a good idea to understand how to enable it and how to query data within it.
In future articles, I’m going to dig into complex queries for threat hunting and investigations.