Skip to content

Event Analytics – Basics

Event Analytics Input Formats

Event

An event is a collection of key-value pairs. Every event must contain a timestamp (_time) and an event description, e.g.:

1
2
3
4
5
6
{
  "_time": "2017-10-01T12:00:00.001Z",
  "text": "Warning: Pressure is to low",
  "text_qc": 0,
  ...
}

Log Files

Event Analytics API can also take input from log files. This requires a preprocessing step. The following example is the log file from a Nanobox:

1
2
3
4
2018-07-02 11:27:42,187 [IN] HandlerThread_33 | c.s.m.a.l.ApplicationLogManager | Main Appender logging structure is initialized. Agent log file name is <MindEdgeRuntimeSystem>.
2018-07-02 11:27:43,202 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?
2018-07-02 11:27:45,205 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?
...

The log messages must be transformed into a format as shown below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
{
  "_time": "2018-07-02 11:27:42.001Z",
  "text": "187 [IN] HandlerThread_33 | c.s.m.a.l.ApplicationLogManager | Main Appender logging structure is initialized. Agent log file name is <MindEdgeRuntimeSystem>."
}
{
  "_time": "2018-07-02 11:27:43.001Z",
  "text": "202 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?"
}
{
  "_time": "2018-07-02 11:27:45.001Z",
  "text": "205 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?"
}
...

Patterns

A pattern consists of multiple event texts along with occurrence boundaries. Using upper and lower limits (maxRepetitions, minRepetitions) for the number of occurrences, a pattern can be configured to allow a distinct number of occurrences or a range. The limits are defined by positive numbers smaller than 999. A pattern can only contain up to 99,999 events. The event text can contain regular expressions, which have to follow java syntax.

Pattern example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
"pattern": [
        {
          "eventText": "Starting turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        },
        {
          "eventText": "Pressure is rising",
          "minRepetitions": 0,
          "maxRepetitions": 10
        },
        {
          "eventText": "Error code: 3.\\d{1,3}",
          "minRepetitions": 0,
          "maxRepetitions": 10
        },
        {
          "eventText": "Stopping turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        }
      ]

Regular Expressions

The following example shows a valid pattern with a regular expression:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
"pattern": [
        {
          "eventText": "Starting turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        },
        {
          "eventText": "Error code: 3.\\d{1,3}",
          "minRepetitions": 0,
          "maxRepetitions": 10
        },
        {
          "eventText": "Stopping turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        }
      ]

The following events fulfill the regular expression Error code: 3.\\d{1,3}:

  • Error code: 302
  • Error code: 305
  • Error code: 3123

The following events do not fulfill the regular expression Error code: 3.\\d{1,3}:

  • Error code: 3aa
  • Error code: 32
  • Error code: 307702
  • Erroor code: 305

Non-Events

A non-event is an event that must not occur in a pattern. The following example illustrates this:

1
2
3
4
5
6
7
pattern: A B C D

non event: X

events: A E B F C G D  match pattern

events: A E B F C X D  does not match pattern

An event can either be part of a pattern or a non-event, but not both.

Any questions left?

Ask the community


Except where otherwise noted, content on this site is licensed under the MindSphere Development License Agreement.