We changed our name from IT Central Station: Here's why

The Math of SIEM Comparison

1814

There are many comparisons and scoring reports like Gartner. But a small part of their scoring is technical capacity. Other comparisons available on the web or magazines are marketing, sales, and presales documents. They do not include extensive technical analysis.

In today’s ever-evolving cybersecurity climate, businesses face more threats than ever before. Finding the right SIEM is crucial in protecting against the latest risks and equipping your organization with a robust security strategy.

A SIEM’s power is in its correlation. 80% of SIEM is a correlation. if you are spending 80 percent of your time within a SIEM tool doing alert review and analysis, then you are on the right track. [SANS Your SIEM Questions Answered]

A detailed comparison of the correlation capacity of SIEM products technically will be given. The comparison based on the most critical correlation and detection capabilities:

  1. Rule Chain (Multi-Stage Rules)
  2. Correlation Logic
  3. List/Watchlist Management
  4. Real-Time Correlation
  5. Cross-Correlation
  6. Correlation Operators
  7. Correlation Field Operators
  8. Correlation Field Restrictions
  9. Machine Learning


Rule Chain (Multi-Stage Rules):

Rule chain is the ability to combine multiple steps (rules) of a use case without any restrictions. This type of rule detects a sequence of events that occur.

Most of the SIEM tools like Micro Focus ArcSight, LogRhythm, Qradar, Securonix, and SureLog support multi-sage rules.

AlienVault, McAfee, FireEye, FortiSIEM, Solarwinds LEM, ManageEngine SIEM are the other SIEM tools that support multi-sage rules with some limits.

McAfee has a restriction on the rule chain capability. For example: "if a firewall admin login has occurred and after this login action there is no configuration change immediately (wait for 15 minutes) but if there is a change in the firewall after this 15 minutes within 12 hours, notify", is not possible with McAfee. It is not possible to develop this type of rule chain because it is not possible to define "wait 15 minutes" then check for "later 12 hours".

McAfee Rule Chain Editor

Since there are two or more actions that require time windows, the 10 minutes must be divided between them. For this example, five minutes is the period for each action. 

Once the unsuccessful attempts have occurred in five minutes, the system begins to listen for a successful login from the same IP source in the next five minutes. .so there is no chance to implement wait logic between actions (rules)

FireEye has the same restrictions as McAfee.

Solarwinds LEM has the same restrictions as McAfee.

ManageEngine SIEM has the same restrictions as Solarwinds LEM. In ManageEngine SIEM, there is no chance to define a new rule type to chain. Also, there are schema fields restrictions to link rule chains.

AlienVault has the same restrictions as Solarwinds LEM. Also, when chaining rules, Alienvault only uses None, Plugin_sid, SRC_IP, DST_IP, SRC_Port, DST_Port, Protocol, and Sendor. There is no way for other schema fields to link rule chains.

FortiSIEM also does not have "wait for 15 minutes" kind of capability to chain rules.

Logpoint does not have this kind of correlation capability.

Rapid7 does not have this kind of correlation capability.

One another requirement when chaining rules, is cross-linking of rule fields. As an example: If a device is the destination of a brute force attack and then this destination device is the source of the port scan, detect this device.

Alienvault only uses SRC_IP, DST_IP, SRC_Port, DST_Port, Protocol rule fields.

McAfee does not support cross-linking of rule fields.

Logpoint does not have this kind of correlation capability.

Rapid7 does not have this kind of correlation capability.

Exabeam and Securonix are UEBA tools. They are not correlation-based solutions.

Correlation Logic:

Rules are discriminators used to find a certain behavior. If their designer knows what it's searching for, they will be invaluable tools. To design a rule without any limits or barriers, the correlation logic of the rule engine must be very powerful and flexible. It is hard to test the correlation logic of the SIEM tools. One of the simplest ways is to try to implement a discriminator use case (correlation rule). For example :

"Detects more than three authentication failures from the same user within five minutes without any successful login in-between."

Micro Focus ArcSight also can detect similar use cases.

If you want to detect this use case with Splunk, it might be possible to do with "transaction" events. But those searches are very taxing in the search head.

Rapid7 and Logpoint have the same issues with Splunk.

AlienVault, FortiSIEM, ManageEngine SIEM, McAfee, Solarwinds LEM could not detect the above use case.

Another test use case is detecting changes. Rapid7 has a change detection capability.

Rapid7 Change Detection Wizard

Qradar also has a change detection capability.

Another example is "Never Seen Before Type Of Rules" [2,3]. While Micro Focus ArcSight, Exabeam, Qradar, Rapid7, Securonix, and SureLog have this capability, AlienVault, McAfee, FireEye, Solarwinds LEM, and ManageEngine do not.

If we continue with the decisive scenarios that are not available in every SIEM product, we can use the below rules:

  1. If a machine bandwidth usage is > 200Mbytes within 5 minutes or If a user to DSTIP bandwidth usage is > 500Mbytes within 10 minutes.
  2. If an account has not been used at least in the last 30 days, notify/lock/delete this account. (This use case is mandatory for FedRAMP Moderate; Control AC-2(3) nd NIST 800-53; Control AC-2(3))

McAfee does not support those types of rules. Please check the below blogs.

The second rule is also not supported by many SIEM products like McAfee, FortiSIEM, LogRhythm, AlienVault, Solarwinds, ManageEngine, and RSA.

Maybe you can develop a query with Splunk and Logpoint. But if you want to detect this immediately, you should consider system resource usage.

List/Watchlist Management:

Micro Focus ArcSight, LogRhythm, Qradar, Securonix, and SureLog - all have a list management feature. 

Both products support simple lists, multi-dimensional lists, complex lists, lists with 20 columns. Also, those products add, delete, modify, list items dynamically, or manually. 

AlienVault:

Dynamic list usage in correlation rules is not supported in AlienVault. It is not possible to develop a rule like If a VPN user connected after business hours and the user is not in VPN white list, alert.

The only way to implement a simple Active Lists is to develop a code.

https://www.alienvault.com/blogs/security-essentials/how-to-use-ossim-usm-active-lists-with-python-scripts

But even if you can develop Python Scripts, there is no key: value, reference set, reference map, multi-dimensional type of lists. 

AlienVault SIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, AlienVault does not support list operators like count, sum, compare, check case sensitivity.

FortiSIEM:

Dynamic list usage in correlation rules is limited to one dimension.

There is no key: value, reference set, reference map, multi-dimensional type of lists. The only available operators are “IN, NOT IN”. Also, the only way of removing items from a watchlist is time-based. Also, FortiSIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, FortiSIEM does not support list operators like count, sum, compare, check case sensitivity.

McAfee:

There is no key: value, reference set, reference map, multi-dimensional, type of lists. McAfee SIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, McAfee SIEM does not support list operators like count, sum, compare, check case sensitivity.

LogPoint:

LogPoint supports two kinds of lists; Static List and Dynamic List. Also LogPoint supports tables, but there is no reference set, reference map, multi-dimensional type of lists.

Also, if you are looking for a GUI for list/watchlist management, LogPoints works over queries. Dynamic lists and table updates are the only query-based. Also, LogPoint SIEM does not support updating multiple lists at the same time (more than one list) by a query. Also, LogPoint SIEM does not support list operators like count, sum, compare, check case sensitivity.

RSA NetWitness Platform:

RSA has a limited list management capability. There is no key: value, reference set, reference map, multi-dimensional, type of lists. Also, RSA SIEM does not support updating multiple lists at the same time (more than one list) by a single rule (only one rule). Also, RSA SIEM does not support list operators like count, sum, compare, check case sensitivity.

There are many other correlation features to check [1] . But without an advanced list/watchlist management, it is not possible to detect advanced attacks.

Real-Time Correlation:

AlienVault, Micro Focus ArcSight, Fireye, FortiSIEM, LogRhythm, ManageEngine SIEM, McAfee, Qradar, RSA NetWitness, Solarwinds LEM has a real-time correlation capability. if you use Splunk ES for real-time detection, you have to consider "Each real-time search unpreemptively locks 1 core on EVERY INDEXER and on your Search Head”.

Elastic also has no real-time correlation feature.

Cross Correlation:

Micro Focus ArcSight, FortiSIEM, LogRhythm, McAfee, Qradar has a cross-correlation capability.

AlienVault cross-correlation can only run on (just) IPS and Vulnerability Scanner logs and the combining on just IP addresses.

RSA NetWitness utilizes ESPER CEP, and there is no GUI for cross-correlation rule development.

Logpoint does not have this kind of correlation capability. Mainly it is a search-based tool.

Rapid7 does not have this kind of correlation capability. Mainly it is a search-based tool.

Exabeam and Securonix are UEBA tools. They are not correlation-based solutions.

Correlation Operators:

Micro Focus ArcSight, LogRhythm, Qradar, SureLog has a correlation operator support feature, such as:

  • And
  • Or
  • Fallowed by Within
  • Not Fallowed by Within

SureLog also has some additional correlation operators like:

  • At the Same Time
  • Before
  • After

McAfee has some missing operators like "At the Same Time", "Before", "Not Fallowed by Within"

Solarwinds LEM documents mention some other correlation limits.. For example you cannot create a rule using “NOT FALLOWED BY” operator."At the Same Time", "Before" are an example of ther missing correlation operators.

Only "AND", "OR" Operator supported. "NOT" Operator is not supported. Also, other operators listed above are not supported.

ManageEngine Eventlog analyzer correlation has only one operator “Fallowed by Within”. Many operators are missing like ”Not Fallowed by Within”.

Also, other operators listed above are not supported like "At the Same Time", "Before".

FortiSIEM does not support "At the Same Time", "Before".

RSA NetWitness does not support "At the Same Time", "Before".

Logpoint does not have this kind of correlation capability. Mainly it is a search-based tool.

Rapid7 does not have this kind of correlation capability. Mainly it is a search-based tool.

Exabeam and Securonix are UEBA tools. They are not correlation-based solutions.

Correlation Field Operators:

Required correlation operators for powerful correlation:

  • Link Fields
  • Check Base64
  • Count Characters
  • In List
  • Not In List
  • Count
  • Sum
  • Regex Matches
  • Matches,
  • Not Maches,
  • Entropy Bigger Than
  • Entropy SmallerThan
  • Is null,
  • Is not null,
  • IP Range Equals,
  • IP Range Not Equals,
  • In list,
  • Not in list,
  • Starts with in list,
  • Starts with in list case insensitive,
  • Not starts with in list,
  • Not starts with in list case insensitive,
  • Contains list key in data,
  • Not contains string in list,
  • Not contains string in list case insensitive,
  • Is contained in string,
  • Regex in list,
  • Check data in regex list,
  • Contains in list,
  • Not contains in list,
  • Contains credit card number

McAfee has some of those correlation operators but less than the above list.

McAfee Operators

AlienVault, FortiSIEM, ManageEngine and Solarwinds LEM does not support most of the above list.

RSA NetWitness utilizes ESPER CEP, and there is no GUI for rule development. ESPER language does not support all of the operators.

Logpoint does not have this kind of correlation capability. Mainly it is a search-based tool.

Rapid7 does not have this kind of correlation capability. Mainly it is a search-based tool.

Exabeam and Securonix are UEBA tools. They are not correlation-based solutions.

Correlation Field Restrictions:

Micro Focus ArcSight, LogRhythm, Qradar has no restrictions on fields. All the available fields on search and report schema will be available for correlation.

McAfee has also some limitations. if I see a user attempt to login to our VPN from two different "regions" within a three-hour window. I have the logic built but in the correlation rules "Advanced Options" I try to set a 'Distinct values' of 2 but the monitored fields only seem to provide a 'Source Geo location' option, and not the ability to select state, region, country, etc.

AlienVault correlation engine has sticky diff restrictions.


Solarwinds LEM does not use all the report fields on correlation. Also, correlation cannot fire on raw log data that is received

ManageEngine SIEM has correlation field restrictions. It is not possible to use all the available reports and search schema in correlation.

Machine Learning:

SureLog, IBM QRadar, Microfocus, LogRhythm, Exabeam, Securonix, NetWitness Platform has NLP/ML/AI features like DGA detection, outlier detections, rarity detection, similarity detection. 

LogPoint uses 3rd party UEBA tool Fortscale (RSA Now).

References:

  1. https://drertugrulakbas.medium.com/detecting-unusual-activities-using-a-next-generation-siem-use-cases-d91f4e24b0f2
  2. https://drertugrulakbas.medium.com/at-the-same-time-siem-operator-be8d6598b7b8
  3. https://www.itcentralstation.com/articles/what-really-matters-when-selecting-a-siem-and-how-to-choose-a-siem-looking-into-the-correlation https://answers.splunk.com/answers/663659/need-help-writing-query-to-alert-if-an-account-has.html
  4. https://techcommunity.microsoft.com/t5/azure-sentinel/azure-sentinel-correlation-rules-the-join-kql-operator/ba-p/1041500
  5. https://gosplunk.com/accounts-deleted-within-24-hours-of-creation/
  6. https://drertugrulakbas.medium.com/siem-for-smb-in-2020-a04e3fe8e98d
  7. https://www.itcentralstation.com/articles/how-to-select-the-right-siem-solution
  8. https://drertugrulakbas.medium.com/ml-ai-is-a-feature-not-a-silver-bullet-and-ueba-questions-d504a6926c4e
  9. https://www.linkedin.com/pulse/ai-cybersecurity-closing-steve-king/
  10. https://towardsdatascience.com/the-limitations-of-machine-learning-a00e0c3040c6
  11. https://www.computerworld.com/article/3466508/the-impact-of-machine-learning-on-security.html
  12. https://drertugrulakbas.medium.com/detecting-unusual-activities-using-a-next-generation-siem-use-cases-part-2-27b201bcc127
Published:
Find out what your peers are saying about Splunk, IBM, Devo and others in Security Information and Event Management (SIEM). Updated: January 2022.
563,780 professionals have used our research since 2012.
11 Comments
author avatarSeckin Demir
Real User

If this article was specific to ANET only and was describing the product and its features, it would be useful for buyers, but it seems to be a different situation. I'll not hide I'm a fan of McAfee (as expected ex McAfee employee) but, I will also be professional. 


I see a simple idea in this article, "It is not possible" for competitor vendors, is that really true? 


I think you can actually say: "Hey, my product is the best, don't spend your time making a PoC with other SIEM/UEBA vendors or partners. Drop the current SIEM/UEBA product and buy my product". This can be more helpful for your sales opportunities! 


But, for the other readers; did you work with any competitor vendors or any certified competitors' partner or any certified engineer of the vendors when preparing/sharing this article? I don't think so. 


Why you are sharing McAfee community links to get support for this article? I think there is no rule or perception of people have to be accepting your idea if a question is not answered on the community portal! Yes, the vendor should care of the community portal but, the vendor also has a Support Portal for their customers! 


I think you should not do and not share anything by yourself about the competitor's product because you are not a certified engineer for the product! If you have, please share with us. But, I'm sure that, no one stops you to talk about others and sharing about their product! Should I write how should be configured a correlation rule for ANET or how should be used ANET? Sorry, this is not my business. 


Let back to your article: Rule Chain perception is not accurate for McAfee and the firewall admin scenario is totally wrong but, even if it is wrong or even if the scenario changed for the customer requirement, it can be easily addressed by McAfee SIEM and I'm definitely sure that the others will find a way also. 


McAfee SIEM can detect this type of scenario and send alerts to SOC/SIEM admins/operators via email, SNMP, telegram, etc. Correlation field operators are also not accurate for McAfee again! Also, your perception is not accurate for the Correlation field restrictions and correlation rule logic on McAfee SIEM because you are not certified and you are not educated for McAfee SIEM. If you are, please share with us. I see only you are right for about the McAfee Watchlist design and ML but, always we can have a workaround! 


If you want to learn how you can do it for all of the scenarios, please buy a McAfee/FortiNet/FireEye... product and submit a support ticket to Support or you can communicate with your Partners to get a professional service. 


Lastly, I think we should focus on the questions, technologies, and needs of the SIEM buyers. We should not say "impossible" or "it is not supported", if this is not our product but, of course, you can talk about ANET because it is your product.

author avatarErtugrul Akbas (ANET)
Top 5LeaderboardReal User

@Seckin Demir If my comments do not satisfy you about the McAfee SIEM, you can check comments from Gartner.
Limited advanced features and add-ons: McAfee lags behind competing SIEM vendors that offer cloud-native SIEM options, ML powered UEBA and SOAR add-on solutions.Requirement for add-ons for a range of cloud environments: Native monitoring of popular SaaS solutions and CIPS by McAfee ESMis limited to Microsoft Office 365, AWS and Microsoft Azure. Other SaaS apps and CIPS require use of MVISION Cloud or an integration with a third-party CASB.Potential impact from sale of enterprise business: In March 2021, McAfee announced the sale of its enterprise business to Symphony Technology Group. This sale may introduce uncertainty for existing customers and potential buyers. Those considering McAfee for SIEM should check its roadmap and future support for McAfee ESM.
Also, you can also look at how the McAfee SIEM product's status in the Gartner Magic Quadrant has changed over the past three years.

author avatarErtugrul Akbas (ANET)
Top 5LeaderboardReal User

@CraigHeartwell, ​thanks for your spelling correction. 


ArcSight acquired Interset for ML. Yes, LogRhythm can handle the logic.


SIEM Comparison table is on my mind for a long time. I published the Turkish version. I need to work to extend it before publishing.

author avatarCraigHeartwell
User

@Ertugrul Akbas Much appreciated, thank you.

author avatarAugusto Barros
Vendor

Ertugrul,


Interesting to see an in-depth comparison, but your comments on Securonix (and Exabeam, BTW) are not really accurate. We provide the ability to build correlation based rules. These solutions started as UEBA, you are correct there, but they have been competing as complete SIEM solutions for a long time by now.

author avatarErtugrul Akbas (ANET)
Top 5LeaderboardReal User

@Augusto Barros my sentence is: "Exabeam and Securonix are UEBA tools. They are not correlation-based solutions". It does not mean that Exabeam and Securonix do not provide the ability to build correlation based rules. I tried to emphasize their UEBA capability.

ITCS user
Guest