Splunk Search Cheat-sheet

Optum Splunk

The Sharepoint site for the Optum Splunk team can be found here which contains information on how to engage their team, upcoming events, best practices, etc.

At a minimum, take a look at the team's best practices documentation

Firewall Logs

Access Firewall Logs in Security Splunk. To gain access to Security Splunk, please see the Onboarding document to find the secure group you need to be a member of. You might want to check the Splunk Best Practices.

Useful Table Formatted Searches

If you want a tabular format, you have to pipe it to table and specify the columns are you looking for. This is normally quite a bit faster than a raw query via the full interface. The fewer columns you specify, the faster your query will run. Some examples:

Useful Searches

Full On-Prem Search

index=sec_n_paloalto_panos dest="161.249.155.20" action!=allowed | table _time dvc src_zone dest_zone src src_translated_ip dest dest_translated_ip transport dest_port application action action_source session_end_reason rule packets_out packets_in vsys_name

Cloud

index=cloud_pa_firewall_sec "data.Action"=allow "data.Application" !=incomplete OR insufficient-data 9bd6db48e294_10.255.96.0_20 "data.DestinationIP"="52.188.247.146" | table _time data.Action data.SourceIP data.DestinationIP data.DestinationPort data.Application reg

Example on-prem outbound

index=sec_n_paloalto_panos  dvc_name="pan-mn053-hosting-01" dest_zone="l3-internet" action=allowed | table _time dvc vsys_name src_zone src_ip dest_zone dest_ip transport dest_port application action rule packets_out packets_in session_end_reason

Suggested for Epic

index="sec_n_paloalto_panos" src_ip="10.150.201.0/24" dvc_name="ohemr-*" | table dvc_name _time src_zone src_ip dest_zone dest_ip transport dest_port app rule

Splunk Search Queries Template

Basic Splunk search template

index=sec_n_paloalto_panos src_ip="1.2.3.4" dest_ip="5.6.7.8" dest_port=443

By default, all Splunk criteria are ANDs. If you need to add in an OR condition, you have to add it. Note that as soon as you use ORs you must use parenthesis to group the conditions:

index=sec_n_paloalto_panos (src_ip="1.2.3.4" OR dest_ip="5.6.7.8") dest_port=443

Please note that ANDs and ORs MUST BE CAPITALIZED. Splunk will silently ignore things it doesn't know, and it is case sensitive.

!!! Hint Useful Additions To display the results in a tabular format. You can pick any columns to display: | table _time dvc src_zone dest_zone src src_translated_ip dest dest_translated_ip transport dest_port application action action_source session_end_reason rule packets_out packets_in vsys_name To display the unique values in a column - for example, unique IPs hitting the firewall - add this onto the query: | stats values(src_ip) as src_ip dc(src_ip) as distinctCountIP

Other indexes (they may require Secure requests)

• index=sec_n_bluecoat_proxy
• index=sec_n_paloalto_panos
• index=sec_n_vmware_firewall
• index=sec_n_misc_products
• index=sec_n_infoblox_nios
• index=sec_n_vmware_firewall
• index=paloalto (OFE: splunk.optumfe.com)

Hot Tips

• Ctrl + \ line breaks to separate commands

Additional

Info Field example

Destination Port dest_port dest_port=80
Rulename  rule rule="ESC-xyz"
firewallname dvcname dvcname="pan-nj777-hub01"
  dvc dvc="pan-nj777-hub01.uhc.com"

Which device logs?

• index: index="look below entries" or *
• action: action="allowed"
• source: src_ip="10.10.10.10" or a.b.c.d/CIDR
• destination: `dest_ip="10.10.10.10" a.b.c.d/CIDR``
• NAT (source): src_translated_ip="x.y.z.a"
• NAT (destination): dest_translated_ip="x.y.z.a"
• Request parameter - request (can be used for fqdn lookup)

Miscellaneous Examples

In order to prevent a search failure with a message like this

“The maximum disk usage quota for the user has been reached.”, follow these practices to reduce the amount of data returned:

• Use the “| head <# of records to return>“ command when executing a search that returns LARGE quantities of detail event logs to limit the number of events returned.
• Use the | fields command to limit the number of fields returned
• Write statistical summary queries when looking for statistical results instead of returning detail events and relying on the field list on the left to provide summary data
• Reuse cached detailed event results by using the | loadjob command, which queries the results of a search in cache ( last 10 minutes ) without querying the index data again. You can then run summary searches against a defined cached dataset and minimize the amount of data added to your cached quota as well as speed up the execution of a series of statistical searches.

Example series of searches using this approach:

• index=sec_n_paloalto_panos earliest=-2minutes
• returned > 20 million events
• | history
• Returns summary of jobs ( search query and results ) in cache
• Use to copy the sid value of the target job results to be used in the next step
• | loadjob $sid ( note: you can use several available values, but I find the sid to be easiest )
• | loadjob $sid |stats count by action_source action

Other examples:

General:

index="sec_n_paloalto_panos" dvc_name="pan-mn053-hosting-01" dest_zone="l3-internet" action=allowed  server_location!="United States" | table  _time dvc vsys_name src_zone src_ip client_location dest_zone dest_ip server_location transport dest_port application action rule packets_out packets_in session_end_reason

Geo-location:

index="sec_n_paloalto_panos" dvc_name="pan-mn053-hosting-01" dest_ip="2.22.230.130" dest_zone="l3-internet" action=allowed | table  _time dvc vsys_name src_zone src_ip client_location dest_zone dest_ip server_location transport dest_port application action rule packets_out packets_in session_end_reason

Counting:

index="sec_n_paloalto_panos" dvc_name="pan-mn053-hosting-01" server_location="China" dest_zone="l3-internet" action=allowed | stats sum(linecount) as Total

index=sec_n_paloalto_panos cef_name=TRAFFIC dvchost=*web* cef_signature=end destinationTranslatedPort=443 OR destinationTranslatedPort=80 act=allow
| dedup src dst
| stats count by src cs4 cs5 dst cs1 destinationTranslatedPort app
| rename cs1 as RuleName
| sort 0 src
| rename cs4 as SrcInterface
| rename cs5 as DstInterface
| rename destinationTranslatedPort as Port

Mods:

cs1!=Office365_Outlook
cs1!=*fice365* cs1!=*utlook*

Redo with just deduped Srcs and removing traffic to UHG ext IP space:

index=sec_n_paloalto_panos cef_name=TRAFFIC dvchost=*web* cef_signature=end destinationTranslatedPort=443 OR destinationTranslatedPort=80 act=allow dst!=149.111.0.0/16 dst!=168.183.0.0/16 dst!=161.249.0.0/16 dst!=198.203.0.0/16
| dedup src
| stats count by src cs4 cs5 dst cs1 destinationTranslatedPort app
| rename cs1 as RuleName
| sort 0 src
| rename cs4 as SrcInterface
| rename cs5 as DstInterface
| rename destinationTranslatedPort as Port

(index=sec_n_bluecoat_proxy "baidu.co.th") OR (index=sec_n_paloalto_panos "baidu.co.th") OR (index=sec_n_misc_products cef_product=MPS "baidu.co.th")
| table _time, dvchost, categoryDeviceGroup, act, src, dst, request, reason, suser | sort -_time

Normal Pano Log Searches:

index=sec_n_paloalto_panos cef_name=TRAFFIC src=10.129.184.70 dst=10.119.122.77 dpt=443
| stats count by src dst dpt app act  cs1 cs4 cs5
| rename cs1 as RuleName
| sort 0 src
| rename cs4 as SrcInterface
| rename cs5 as DstInterface
| rename dpt as Port

Search Parameters for use in the dashboard taking input from the user for Source (src=$src_ip$), Destination (dst=$dst_ip$), and Port (dpt=$prt_nmbr$)

index=sec_n_bluecoat_proxy  src=$src_ip$
| head 500
| stats count by _time src dest dest_port action app category user cs_threat_risk url
| rename action as Action
| rename cs_threat_risk as "Threat Risk Level"
| rename category as "Category"
| rename src as "Source IP"
| rename dest as "Destination"
| rename dest_port as "Port"
| rename url as "URL Request"

index=sec_n_paloalto_panos action!=allowed src=10.86.165.153 | head 1500
| stats count by _time src dest_ip dest_port app action rule vsys_name dvc_name
| rename src as "Source IP"
| rename dest_ip as "Destination IP"
| rename app as "Palo App"
| rename rule as "Rule Name"
| rename action as Action
| rename packets_out as "Packets Sent"
| rename packets_in as "Packets Received"
| rename dest_port as Port
| rename dvc_name as "Firewall Name"
| rename vsys_name as "Vsys Name"

index=sec_n_paloalto_panos action!=allowed (dest_ip=168.183.250.248 OR dest_ip=168.183.249.248) | head 1500
| stats count by _time src dest_ip transport dest_port app action rule vsys_name
| rename src as "Source IP"
| rename dest_ip as "Destination IP"
| rename app as "Palo App"
| rename rule as "Rule Name"
| rename action as Action
| rename packets_out as "Packets Sent"
| rename packets_in as "Packets Received"
| rename dest_port as Port
| rename dvc_name as "Firewall Name"
| rename vsys_name as "Vsys Name"
|rename transport as "Protocol"

index=sec_n_paloalto_panos type=THREAT action!=allowed src=$src_ip$ dest_ip=$dst_ip$ dest_port=$prt_nmbr$
| stats count by _time src dest_ip dest_port app action rule vsys_name threat threat_name severity dvc_name
| rename src as "Source IP"
| rename dest_ip as "Destination IP"
| rename app as "Palo App"
| rename rule as "Rule Name"
| rename action as Action
| rename packets_out as "Packets Sent"
| rename packets_in as "Packets Received"
| rename dest_port as Port
| rename dvc_name as "Firewall Name"
| rename vsys_name as "Vsys Name"
| rename threat as "Threat"
| rename threat_name as "Threat Name"
| rename severity as "Severity"

index=sec_n_vmware_firewall cef_product="NSX Firewall" src=$src_ip$ dst=$dst_ip$ dpt=$prt_nmbr$
| stats count by src dst dpt proto act
| rename src as "Source IP"
| rename dst as "Destination IP"
| rename app as "Palo App"
| rename cs1 as "Rule Name"
| rename act as Action
| rename cs4 as SrcInterface
| rename cs5 as DstInterface
| rename dpt as Port

OLD

index=sec_n_bluecoat_proxy  cef_name=TUNNELED src=$src_ip$
| stats count by _time src dhost request act app cat cef_name cs4 cs2
| rename act as Action
| rename cs4 as "URL Category"house
| rename cs2 as "Threat Risk Level"
| rename cat as "Traffic Type"
| rename src as "Source IP"
| rename dhost as "Destination IP"
| rename request as "URL Request"

index=sec_n_paloalto_panos  dest_ip=149.111.149.190
| stats count by _time src dest_ip transport dest_port app action rule vsys_name
| rename src as "Source IP"
| rename dest_ip as "Destination IP"
| rename app as "Palo App"
| rename rule as "Rule Name"
| rename action as Action
| rename packets_out as "Packets Sent"
| rename packets_in as "Packets Received"
| rename dest_port as Port
| rename dvc_name as "Firewall Name"
| rename vsys_name as "Vsys Name"
|rename transport as "Protocol"