Tag Archive: Sentinel

SEC-LABS R&D > Sentinel

Quick tip – Country Codes

2023-11-14 0 Comments

All countries has an ISO code, described in ISO 3166 is an international standard.
These codes are used throughout the IT industry by computer systems and software to make it easier to identify a country.

It has multiple formats and they country codes are presented in the following formats: Alpha-2 (2 characters), Alpha-3 (3 characters) and Numeric (3 digits).

In the data from some logs like SigninLogs and IdentityLogonEvents the country is presented as Alpha-2. We realized pretty quick is that some 2-characters country codes are difficult to remember. As in below, picture it could be difficult to know all these countries.

We have been using this for a long time and thought it might be something others can use as well.

So to solve this I created a csv file and placed on github:

https://raw.githubusercontent.com/mattiasborg82/Hunting/main/General/cc.txt

To be able to join our data with this file we can use the external data operator in Kusto

Since it’s a CSV file, we can make it more usable by split the rows on comma

To to build the full use-case for this, we join it with our SigninLogs (or other logs that uses the country code)

Copy friendly code

let CountryCodes = externaldata (CountryCode:string)
[
 @"https://raw.githubusercontent.com/mattiasborg82/Hunting/main/General/cc.txt"
]
with(ignoreFirstRecord=true);
SigninLogs
| where isnotempty(Location)
| join kind=leftouter (
    CountryCodes
    | extend Country = tostring(split(CountryCode, ",")[0]),
              Location = tostring(split(CountryCode, ",")[1])
    | project-away CountryCode)
on Location
| summarize count() by Country,UserDisplayName

This can be used further to combine with conditional access blocks showing potential credential leak

Happy Hunting!

Detect, Security

, , , ,

Threat Hunting and the New Hunts in Sentinel

2023-08-22 0 Comments

Establishing a Proactive Hunting program is something which is useful and necessary today.

From working with proactive threat hunting for a long time, from when data was not available at your fingertips, things has become a lot easier in the era of SIEM and over the last years, EDR and XDR.

The technical part is the easy one. The process of establishing your hunting and connect it with existing processes is usually what’s difficult.

What is proactive Hunting?

First, what is threat hunting?

To dive in to this topic I want to point out activities that are somewhat related

The custom detections or scheduled rules is pretty clear what it is. but Tasks is something which sits in between Proactive Threat Hunting and Custom Detections.
Tasks are things discovered in Hunting, and data of interest but cannot be set as a custom detection as of yet.

The reasons could be many, but for example to noisy data and no good correlation values or near-time events that can be used to reduce False-Positives. Even though we have finalized our hypothesis from threat hunting we might want to follow-up on the events, maybe on a daily basis, but we cant have it scheduled since it will give an incident fatigue.

To summarize tasks, it’s queries that might make it to a detection but for now we run it automatically or manual and do manual review on the result on a daily schedule.

To iterate back to threat hunting before we take a look at the hunts feature in Sentinel

Threat Hunting can be divided into 2 main pillars; Proactive and Reactive hunting

Proactive Threat Hunting is when you don’t know something is going on, like playing hide-n-seek, except for that you don’t know anyone is playing with you

Reactive Threat Hunting is post breach/post incident and you use threat hunting to find the outer boundaries of the incident, which could be other devices communicating with a specific IP, list all process communicating, find the same processes on other devices, and then which IP’s they are communicating with.

In the EDR era or now, the XDR era, threat hunting becomes easier on a technical level. The data collection happens automatically for many workloads and not only collected, it’s streamed.

With the power of Kusto Query Language, you can do advanced aggregations, anomaly calculations and visualizations to truly crunch and bend your data.

In Proactive Threat hunting, you will start your assignment by defining your Hypothesis, which could be something like “I would like to see if any local users have been added outside the process”

Then you check what ever data you need to discover such activities (DeviceEvents/DeviceProcessEvents hint hint) and you continue to develop the query for is and document your result.

Here is what makes Hunts feature so great. It actually allows for process integration, like we have in Microsoft 365 Defender where we can create incidents based on the results and have it handled by the incident process.

Hunts in Sentinel

Common use cases:

  • Proactively hunt based on specific MITRE techniques, potentially malicious activity, recent threats, or your own custom hypothesis.
  • Use security-researcher-generated hunting queries or custom hunting queries to investigate malicious behavior.
  • Conduct your hunts using multiple persisted-query tabs that enable you to keep context over time.
  • Collect evidence, investigate UEBA sources, and annotate your findings using hunt specific bookmarks.
  • Collaborate and document your findings with comments.
  • Act on results by creating new analytic rules, new incidents, new threat indicators, and running playbooks.
  • Keep track of your new, active, and closed hunts in one place.
  • View metrics based on validated hypotheses and tangible results.

In Sentinel go to Hunting and Hunts

Here is the list of previous hunts.

If you select New Hunt we can create a new

You can add the Hypothesis and choose if it’s Validated or not (which can be set later in the process by another Threat hunter)

When we have our new Hunting campaign, we can add queries

Adding Tactics and techniques and map entities

It’s possible to create incidents from the results to map to the incident process, and we can also start automated playbooks (entity-based) from the entity pane.

Summary

There are so many details on this feature and it has some many capabilities. The best part is that it has the process of hunting as its core. Now it’s easier to deliver threat hunting, as a service, as a consultant or if you work internally for an organization.

Happy Hunting

Detect, Respond, Security

, , ,

Microsoft Sentinel Parsing tips – Whitespace control

2023-04-25 0 Comments

This post will be a part of a multiple posts to cover data parsing in Microsoft Sentinel.

Intro

Kusto is a powerfull query language and easy to adopt.

Even if Kusto is very powerfull, working with custom log sources is, sometimes, a mess. Some parsers requires more effort and some are very simple.

In general, when it’s possible to use operators like “parse” (link) function or “parse-kv” (link) it’s very welcome. However, the reality has a different challenge for us.

In this post we want to share a quick pro tip to solve the mystic of hidden whitespaces

The challenge of whitespaces

Whitespaces ” ” exists everywhere, the challenge is how it’s presented in log analytics.

Log analytics does a lot for the user in terms of nicely present data. It actually removes duplicate whitespaces, as well as leading and trailing whitespace.

This could result in problems like failing parsers, regex and string operators like “==”, “startswith”, “endswith” etc will fail. Especially if it’s not consistent.

Marking the string in the output view does not show the extra whitespaces

Copying the text and paste into a text-editor will not show it either like in below example where we copied the output into VS Code (we can only see one dot to show one whitespace between foo and bar)

However, the double whitespaces are interpreted during execution, and it’s only in the presentation view the extra is removed. As in below example, we used split on ” ” to show the existence of the double whitespace.

When working with multiple log sources you don’t want to search and see if they exist (which may change during the log source life cycle), you rather want a way to always make the log to look good in your parser.

Solution

To properly address this (if there aren’t any good ways to change the audit settings of the system sending the logs)

To handle the duplicate white spaces we use the replace_regex function (link here) and use the whitespace “\s” with the quantifier “+” which means one or multiple times and replace it with a space ” “.

This will search for spaces (one or more) and replace it with a single, because we don’t want to remove single spaces. And by using the same column name “SyslogMessage” we will actually reuse the same column for our clean output.

Please note that this will not change the message in the database, only during execution.

Doing this gives us the following output.

The next step is that we want to remove the leading and trailing whitespaces. If we for instance expect the first character to be a value, the leading whitespace could make our parser to fail or an analytic rule.

We have seen occasions where this happens from time to time and not all messages in a log source.

To fix the leading and trailing whitespaces we use another regex to look for start of string and end of string. But this time we want to replace with “nothing/null” which is why we can’t use this regex in the first cleaning.

In the second run we use the same column name again to cleanup the SyslogMessage. There is a best practice to always keep the original message, however, this is to solve an error from the log source and not to alter the SyslogMessage.

The regex starts with an anchor “^” to define the start of the string and followed by a whitespace “\s” since we cleaned all double whitespaces we don’t need to use the quantifier. To handle the trailing whitespace we use the OR operand “|” and check for a whitespace “\s” followed by the anchor “$” to determine the end of the string. If we get any hits it will be replaced with null and we have a clean string.

By adding these 2 lines of code to the parser, we will avoid running into strange issues which could take some time to troubleshoot.

//Sample
CustomLogSource_CL
| extend SyslogMessage = replace_regex(SyslogMessage,@"\s+",@" ") //Remove duplicate whitespaces
| extend SyslogMessage = replace_regex(SyslogMessage,@"^\s|\s$",@"") //Remove leading and trailing whitespaces

Happy Hunting!

Detect, Events, Security

, , , , , , ,

Near-Real-Time analytic rules in Microsoft Sentinel

2021-11-08 0 Comments

NRT Rules are hard-coded to run once every minute and capture events ingested in the preceding minute.

This is for faster detection and response opportunity.

Considerations

  • No more than 20 rules can be defined per customer at this time
  • As this type of rule is new, its syntax is currently limited but will gradually evolve. Therefore, at this time the following restrictions are in effect:
    • The query defined in an NRT rule can reference only one table. Queries can, however, refer to multiple watchlists and to threat intelligence feeds.
    • You cannot use unions or joins.
    • Because this rule type is in near real time, we have reduced the built-in delay to a minimum (two minutes).
    • Since NRT rules use the ingestion time rather than the event generation time (represented by the TimeGenerated field), you can safely ignore the data source delay and the ingestion time latency (see above).
    • Queries can run only within a single workspace. There is no cross-workspace capability.
    • There is no event grouping. NRT rules produce a single alert that groups all the applicable events.

There is a technical limit which blocks union, join etc.

For further information about Near-Real-Time, NRT, analytic rules, please visit:

https://docs.microsoft.com/en-us/azure/sentinel/near-real-time-rules

Happy Hunting!

Detect, Security

, ,

Becoming a Sentinel Notebooks Ninja – training links

2021-09-09 0 Comments

Do you want to learn more about Sentinel Notebooks (built on Jupyter Notebooks)? Microsoft have released a set of trainings to skill up in the area

Notebooks can be useful for cross tenant hunting and also cross product and multiple data sources if needed.

They can also be interactive in terms of a manual playbook with steps mixed with queries and graphs which would make it easy to follow through.

Sorry for the short blog post, but this one is about sharing content

Happy Hunting!

Detect, Security

, , ,

Microsoft 365 Defender connector for Azure Sentinel in public preview

2020-11-10 0 Comments

365 defender connector

A new connector for Microsoft 365 Defender is in public preview in Azure Sentinel. This connector makes it possible to ingest the hunting data into Sentinel

Currently, the Defender for Endpoint Data is available

To enable

  • Go to you Azure Sentinel Instance and select Connectors
  • Search for Microsoft 365 Defender
365 defender connector
  • Click Open Connector Page
  • Select which Events you want to ingest
threat hunting data
  • Click Apply Changes

Example queries

//Registry events
DeviceRegistryEvents
| where ActionType == "RegistryValueSet"
| where RegistryValueName == "DefaultPassword"
| where RegistryKey has @"SOFTWAREMicrosoftWindows NTCurrentVersionWinlogon"
| project Timestamp, DeviceName, RegistryKey
| top 100 by Timestamp
//Process and Network events
union DeviceProcessEvents, DeviceNetworkEvents
| where Timestamp > ago(7d)
| where FileName in~ ("powershell.exe", "powershell_ise.exe")
| where ProcessCommandLine has_any("WebClient",
"DownloadFile",
"DownloadData",
"DownloadString",
"WebRequest",
"Shellcode",
"http",
"https")
| project Timestamp, DeviceName, InitiatingProcessFileName,
InitiatingProcessCommandLine,
FileName, ProcessCommandLine, RemoteIP, RemoteUrl, RemotePort, RemoteIPType
log view

If we look at the tables we can see the new created tables

table view

More information about the data in these tables is available in this post https://blog.sec-labs.com/2018/06/threat-hunting-with-windows-defender-atp/

For further reading:

News, Security

, , , , , ,

Azure Sentinel is now GA

2019-09-25 0 Comments

Azure Sentinel—the cloud-native SIEM that empowers defenders is now generally available

azure sentinel

Some of the new features are:

  • Workbooks are replacing dashboards, providing for richer analytics and visualizations
  • New Microsoft and 3rd party connectors

Detection and hunting:

  • Out of the box detection rules: The GitHub detection rules are now built into Sentinel.
  • Easy elevation of MTP alerts to Sentinel incidents.
  • Built-in detection rules utilizing the threat intelligence connector.
  • New ML models to discover malicious SSH access, fuse identity, and access data to detect 35 unique threats that span multiple stages of the kill chain. Fusion is now on by default and managed through the UI
  • Template playbooks now available on Github.
  • New threat hunting queries and libraries for Jupyter Notebooks

Incidents:

  • The interactive investigation graph is now publicly available.
  • Incidents support for tagging, comments, and assignments, both manually and automatically using playbooks.

MSSP and enterprise support:

  • Azure Lighthouse for multi-tenant management
  • RBAC support

For further information:

Pricing: https://azure.microsoft.com/en-us/pricing/details/azure-sentinel/
Product page: https://azure.microsoft.com/en-us/services/azure-sentinel/
Documentation: https://docs.microsoft.com/en-us/azure/sentinel/

Happy Hunting

Detect, Events, Respond, Security

, , , ,

Audit Scheduled tasks using Azure Sentinel

2019-03-19 1 Comment

Azure Sentinel is a powerful cloud based SIEM solution.
This blog series will be on how to work with Sentinel.

It will be example based on different solutions which we might run into.

This first post is about how you can work with logs and get insight in Scheduled Tasks as a way for attackers to persist in your network

For further information regards Sentinel, visit https://azure.microsoft.com/en-us/services/azure-sentinel/

Scheduled Tasks

By default there are no events created if someone creates or modifieds a scheduled task. To enable logging you have to enable logging of object access.

To view current settings, use the following command:

auditpol.exe /get /category:*

Only Success is required for this. This enables us to get the event 4698

To enable logging, create a new GPO and assign the following settings (depending if you want success/failure or only success)

You also have to configure your agents to send log to your workspace, you can download the agent from the Azure Sentinel workspace / <workspace name> / Advanced Settings

Otherwise, you can add the Sentinel workspace to your existing agents

$Agent = New-Object -ComObject AgentConfigManager.MgmtSvcCfg
$ID= "<WorkspaceID>"
$Key = "<key>"
$Agent.AddCloudWorkspace($ID,$Key)
restart-service HealthService

In Azure Sentinel – Data connectors, configure Security Events

Azure Sentinel Connectors

Verify heartbeats from computers


Heartbeat | summarize argmax(TimeGenerated, *) by Computer


So now we have logs from 2 computers and now we want to query Scheduled Tasks

A simple way is to just query the EventID

sentinel

We can use project to format our table but we still want to get information about the tasks that were created to get a better overview


According to documentation we can use Parse operator into one or more calculated columns

https://docs.microsoft.com/en-us/azure/kusto/query/parseoperator

//Example
SecurityEvent
| where EventID == "4698"
| parse EventData with * '"SubjectUserName">' SubjectUserName '<' * '"SubjectDomainName">' SubjectDomainName '<' *

This query will

  • Select all events where eventid=4698
  • parse the column event data and look for ‘”SubjectUserName”>’
  • Put everyting to a column named SubjectUserName until character ‘<‘
  • the wildcard will run the samething again
  • Continue parsing until ‘”SubjectDomainName”>’
  • Put everything into column SubjectDomainName until character ‘<‘
sentinel

To continue this to get some really useful information we continue to parse the content until we get everything we need

//Sec-Labs Demo - Sentinel Hunting for Scheduled Tasks Persistance
let start=datetime("2019-03-12T19:39:47.762Z");
let end=datetime("2019-03-19T22:39:47.762Z");
SecurityEvent
|where TimeGenerated > start and TimeGenerated < end
| where EventID == "4698"
| parse EventData with * '"SubjectUserName">' SubjectUserName '<' * '"SubjectDomainName">' SubjectDomainName '<' * '"TaskName">\\' TaskName '<' * 'Author>' Author '<' * '<Command>' SchedCommand '</Command' * 'Arguments>' SchedArgs '</Arguments' * 'WorkingDirectory>' SchedDir '&' *
| where isnotempty (SubjectUserName) 
| project TimeGenerated,SubjectUserName,Computer,Activity,SubjectDomainName,TaskName,SchedCommand,SchedArgs,SchedDir
| project-rename CreatedBy = SubjectUserName
hunting queries

To rename columns, you can use project-rename <new name> = <old column name>

Happy Hunting!

Detect

, , , , , ,