Security Information and Event Management

Skip to end of metadata
Go to start of metadata

Overview

Security information and event management ( SIEM ) software products and services combine security information management (SIM) and security event management (SEM). SIEM systems receive security alerts from any number of systems, including application and network hardware, and provide real-time analysis of these alerts. Two of the leading protocols used in SIEM systems are Log Event Extended Format (LEEF), and Common Event Format (CEF). You can easily pass Gigya events over to a SIEM server using Identity Sync, Gigya's ETL solution, while flexibly selecting the events you wish to include and the severity assigned to each event.

LEEF 

The Log Event Extended Format (LEEF) is a customized event format for IBM Security QRadar. Documentation regarding the format can be found here for v1, and here for v2.  

The structure of LEEF v.1 is: Version|Vendor|Product|Version|EventID|key=value key=value key=value key=value

The structure of LEEF v.2 is: Version|Vendor|Product|Version|EventID|DelimiterCharacter|key=value key=value key=value key=value


*In LEEF 1.0, key=value pairs require TAB separation. In LEEF 2.0, It is possible to use ^ or | to separate the key=value pairs, by using the Delimiter Character. Note that the header parameters are always separated using a pipe delimiter. 

Examples: 

LEEF:1.0|Microsoft|MSExchange|2007|7732| 

LEEF:1.0|Microsoft|MSExchange|2007|Logon Failure|

LEEF:2.0|Vendor|Product|Version|EventID|^|

 

  • Version: an integer value that identifies the major and minor version of the LEEF format used for the event.
  • Vendor: a text string that identifies the vendor or manufacturer of the device sending the syslog event in LEEF format.
  • Product: a text string that identifies the product sending the event log to QRadar. 
  • Version: a string that identifies the version of the software or appliance sending the event log.
  • EventID:  a unique identifier for an event in the LEEF header. The purpose of the EventID is to provide a fine grain, unique identifier for an event without the need to examine the payload information. An EventID can contain either a numeric identified or a text description. In the above examples, the event ID is numeric in the first example, and a string in the second. 

    • The value of the event ID must be consistent, and static across products that support multiple languages. If your product supports multi-language events, you can use a numeric or textual value in the EventID field, but it must not be translated when the language of your appliance or application is altered.
    • The EventID field cannot exceed 255 characters.
  • Delimiter Character: in LEEF v.2, specify the delimiter that separates between key-value pairs in the event attribute.
  • Event attribute: a set of key-value pairs that provide detailed information about the security event. Each event attribute must be separated by a tab delimiter, but the order of attributes is not enforced.

CEF

Common Event Format (CEF) can be readily adopted by vendors of both security and non-security devices. This format contains the most relevant event information, making it easy for event consumers to parse and use them.

Documentation regarding the format can be found here

The structure of CEF is: 

CEF:Version|Device Vendor|Device Product|Device Version|Signature ID|Name|Severity|Extension

  • Version: an integer and identifies the version of the CEF format. Event consumers use this information to determine what the following fields represent.
  • Device Vendor, Device Product and Device Version are strings that uniquely identify the type of sending device. No two products may use the same device-vendor and device-product pair. There is no central authority managing these pairs. Event producers have to ensure that they assign unique name pairs.
  • Signature ID: a unique identifier per event-type. This can be a string or an integer. Signature ID identifies the type of event reported. In the intrusion detection system (IDS) world, each signature or rule that detects certain activity has a unique signature ID assigned. This is a requirement for other types of devices as well, and helps correlation engines deal with the events.
  • Name: a string representing a human-readable and understandable description of the event. The event name should not contain information that is specifically mentioned in other fields. For example: “Port scan from 10.0.0.1 targeting 20.1.1.1” is not a good event name. It should be: “Port scan”. The other information is redundant and can be picked up from the other fields.
  • Severity: an integer and reflects the importance of the event. Only numbers from 0 to 10 are allowed, where 10 indicates the most important event. See below for more information on resolving the event severity.
  • Extension:  a collection of key-value pairs. The keys are part of a predefined set. An event can contain any number of key-value pairs in any order, separated by spaces (“ “). If a field contains a space, such as a file name, this is valid and can be logged in exactly that manner, as shown below:
    fileName=c:\Program<space>Files\ArcSight is a valid token.

 

Gigya Implementation

Any event recorded in Gigya's Audit Log can be sent to a SIEM system in either a LEEF or CEF formatted file, using IdentitySync, Gigya's ETL (extract, transform, load) platform. An extract job should be configured to run at a relatively high frequency, providing a constant feed of near real-time data to the SIEM server. Any of the audited events can be used to create the SIEM LEEF and CEF Event logs. The full list of events can be found in the Audit Log documentation

For example, to export Gigya data in LEEF format, the following IdentitySync components could be used: 

  • datasource.read.gigya.audit: select which events should be read from Gigya's Audit Log. For example: “endpoint=”accounts.isAvailableLoginID”. In addition, since LEEF requires a version number, read the information from the httpReq.SDK parameter: 

Note that you can read additional fields from the audit log, and pass them to the SIEM server in LEEF format. For example, read the ip field returned from audit.search , and rename it to src

  • field.rename: rename Gigya audited fields to match LEEF:
  • field.add: add fields that are required for the relevant format, such as "Vendor": 
  • field.evaluate: use JEXL to create key=value pairs under the event attribute, where the keys are hard-coded, and the values derived from the fields that were extracted from the audit log: 

    {
       "id": "field.evaluate",
       "type": "field.evaluate",
       "params": {
        "fields": [
         {
          "field": "Event attribute",
          "expression": "'src='+src+' devTime='+devTime+' devTimeFormat='+devTimeFormat"
         }
        ],
        "language": "jexl"
       },
       "next": [
        "dsv"
       ]
      }
  • file.format.dsv: format the header to the required format, using a pipe '|' delimiter. 
  • datasource.write.sftp:  upload the file to a dedicated location for processing by the SIEM system.
 Click to see the source code of this flow
 {
 "siteId": ...,
 "id": "...",
 "name": "SIEM",
 "description": "...",
 "lastRuntime": "2018-02-28T16:53:01.343Z",
 "steps": [
  {
   "id": "audit",
   "type": "datasource.read.gigya.audit",
   "params": {
    "select": "httpReq.SDK,endpoint",
    "deltaField": "@timestamp"
   },
   "next": [
    "rename"
   ]
  },
  {
   "id": "rename",
   "type": "field.rename",
   "params": {
    "fields": [
     {
      "sourceField": "httpReq.SDK",
      "targetField": "Version"
     },
     {
      "sourceField": "endpoint",
      "targetField": "EventID"
     },
     {
      "sourceField": "ip",
      "targetField": "src"
     },
     {
      "sourceField": "@timestamp",
      "targetField": "devTime"
     }
    ]
   },
   "next": [
    "field.add"
   ]
  },
  {
   "id": "field.add",
   "type": "field.add",
   "params": {
    "fields": [
     {
      "field": "Vendor",
      "value": "SAP"
     },
     {
      "field": "Product",
      "value": "SAP Hybris Identity"
     },
     {
      "field": "LEEFVersion",
      "value": "LEEF 1.0"
     },
     {
      "field": "devTimeFormat",
      "value": "ISO 8601"
     }
    ]
   },
   "next": [
    "field.evaluate"
   ]
  },
  {
   "id": "field.evaluate",
   "type": "field.evaluate",
   "params": {
    "fields": [
     {
      "field": "Event attribute",
      "expression": "'src='+src+' devTime='+devTime+' devTimeFormat='+devTimeFormat"
     }
    ],
    "language": "jexl"
   },
   "next": [
    "dsv"
   ]
  },
  {
   "id": "dsv",
   "type": "file.format.dsv",
   "params": {
    "fileName": "file.csv",
    "columnSeparator": "|",
    "quoteFields": false,
    "escapeCharacter": "\"",
    "writeHeader": false,
    "lineEnd": "\n",
    "createEmptyFile": false,
    "columns": [
     "LEEFVersion",
     "Vendor",
     "Product",
     "Version",
     "EventID"
    ]
   },
   "next": [
    "sftp"
   ]
  },
  {
   "id": "sftp",
   "type": "datasource.write.sftp",
   "params": {
    "host": "...",
    "username": "...",
    "password": "...",
    "port": 22,
    "temporaryUploadExtension": false
   }
  }  
 ],
 "apiKey": "...",
 "updatedByName": "...",
 "updatedByEmail": "..."
}


Assigning a Severity

Severity is a required parameter in CEF, and optional (but recommended) for LEEF. You can apply any business logic to the SIEM IdentitySync flow. For example, you can decide on a closed set of events you wish to read from the audit log and report to the SIEM server, read each event in a separate dataflow, then assign a score depending on the result of the event and the logic you wish to apply. 

In the following example, we use the where  parameter of datasource.read.gigya.audit to extract accounts.login events and the error code of that call (in this example, an error code of 0 indicates successful login, so we query anything that is different from "0"). 

The audit log "where" parameter could read: 

"where": "endpoint='accounts.login' AND errCode <> 0",

So that anything that is not a successful login would be assigned the same degree of severity (in the following dataflow steps), or you could be more specific and retrieve only the error code that indicates a failed login:

"where": "endpoint='accounts.login' AND errCode = 403042",

You can add further nuance: for example, query the UID parameter as well, and if the UID of a failed login attempt belongs to an admin user, increase the degree of severity associated with the event.  

 

  • No labels