idstools.unified2 module¶
Unified2 record and event reading.
Unified2 is a file format used by the Snort and Suricata IDS engines for logging events.
For more information on the unified2 file format see:
usage: from idstools import unified2
-
class
idstools.unified2.
Aggregator
[source]¶ Bases:
object
Deprecated: Applications requiring the aggregation of packets and extra data with an event should implement their own aggregation logic.
A class implementing something like the aggregator pattern to aggregate records until an event can be built.
-
class
idstools.unified2.
Event
(event)[source]¶ Bases:
dict
Event represents a unified2 event record with a dict-like interface. The unified2 file format specifies multiple types of event records, idstools normalizes them into a single type.
Fields:
- sensor-id
- event-id
- event-second
- event-microsecond
- signature-id
- generator-id
- signature-revision
- classification-id
- priority
- source-ip
- destination-ip
- sport-itype
- dport-icode
- protocol
- impact-flag
- impact
- blocked
- mpls-label
- vlan-id
Deprecated: Methods that return events rather than single records will also populate the fields packets and extra-data. These fields are lists of the
Packet
andExtraData
records associated with the event.
-
class
idstools.unified2.
EventDecoder
(fields)[source]¶ Bases:
idstools.unified2.AbstractDecoder
Decoder for event type records.
-
class
idstools.unified2.
ExtraData
(*fields, **kwargs)[source]¶ Bases:
dict
ExtraData represents a unified2 extra-data record with a dict like interface.
Fields:
- event-type
- event-length
- sensor-id
- event-id
- event-second
- type
- data-type
- data-length
- data
-
class
idstools.unified2.
ExtraDataDecoder
(fields)[source]¶ Bases:
idstools.unified2.AbstractDecoder
Decoder for extra data type records.
-
class
idstools.unified2.
Field
(name, length, fmt=None)[source]¶ Bases:
object
A class to represent a field in a unified2 record. Used for building the decoders.
-
fmt
¶ Builds a format string for struct.unpack.
-
-
class
idstools.unified2.
FileEventReader
(*files)[source]¶ Bases:
object
Deprecated: Event readers have been deprecated due to the deprecation of the Aggregator.
FileEventReader reads records from one or more filenames and aggregates them into events.
Parameters: files... – One or more files to read events from. Example:
reader = unified2.FileEventReader("unified2.log.1382627941", "unified2.log.1382627966) for event in reader: print(event)
-
class
idstools.unified2.
FileRecordReader
(*files)[source]¶ Bases:
object
FileRecordReader reads and decodes unified2 records from one or more files supplied by filename.
Parameters: files... – One or more filenames to read records from. Example:
reader = unified2.RecordReader("unified2.log.1382627941", "unified2.log.1382627966) for record in reader: print(record)
-
class
idstools.unified2.
Packet
(*fields, **kwargs)[source]¶ Bases:
dict
Packet represents a unified2 packet record with a dict-like interface.
Fields:
- sensor-id
- event-id
- event-second
- packet-second
- packet-microsecond
- linktype
- length
- data
-
class
idstools.unified2.
PacketDecoder
(fields)[source]¶ Bases:
idstools.unified2.AbstractDecoder
Decoder for packet type records.
-
class
idstools.unified2.
RecordReader
(fileobj)[source]¶ Bases:
object
RecordReader reads and decodes unified2 records from a file-like object.
Parameters: fileobj – The file-like object to read from. Example:
fileobj = open("/var/log/snort/merged.log.1382627987", "rb") reader = RecordReader(fileobj): for record in reader: print(record)
-
class
idstools.unified2.
SpoolEventReader
(directory, prefix, follow=False, delete=False, bookmark=False)[source]¶ Bases:
object
Deprecated: Event readers have been deprecated due to the deprecation of the Aggregator.
SpoolEventReader reads records from a unified2 spool directory and aggregates them into events.
Required parameters:
Parameters: - directory – Path to unified2 spool directory.
- prefix – Filename prefix for unified2 log files.
Optional parameters:
Parameters: - follow – Set to true to follow the log files. Reading will wait until an event is available before returning.
- delete – If True, unified2 files will be deleted when reading has moved onto the next one.
- bookmark – If True, the reader will remember its location and start reading from the bookmarked location on initialization.
Example:
reader = unified2.SpoolEventReader("/var/log/snort", "unified2.log") for event in reader: print(event)
-
class
idstools.unified2.
SpoolRecordReader
(directory, prefix, init_filename=None, init_offset=None, follow=False, rollover_hook=None)[source]¶ Bases:
object
SpoolRecordReader reads and decodes records from a unified2 spool directory.
Required parameters:
Parameters: - directory – Path to unified2 spool directory.
- prefix – Filename prefix for unified2 log files.
Optional parameters:
Parameters: - init_filename – Filename open on initialization.
- init_offset – Offset to seek to on initialization.
- follow – Set to true if reading should wait for the next record to become available.
- rollover_hook – Function to call on rollover of log file, the first parameter being the filename being closed, the second being the filename being opened.
Example with following and rollover deletion:
def rollover_hook(closed, opened): os.unlink(closed) reader = unified2.SpoolRecordReader("/var/log/snort", "unified2.log", rollover_hook = rollover_hook, follow = True) for record in reader: print(record)
-
iter_next
()[source]¶ Return the next record or None if EOF.
If in follow mode and EOF, this method will sleep and and try again.
Returns: A record of type Event
,Packet
,ExtraData
orUnknown
if the record is of an unknown type.
-
class
idstools.unified2.
Unified2Bookmark
(directory=None, prefix=None, filename=None)[source]¶ Bases:
object
Class to represent a “bookmark” for unified2 spool directories.
-
class
idstools.unified2.
Unknown
(record_type, buf)[source]¶ Bases:
object
Class to represent an unknown record type.
In the unlikely case that a record is of an unknown type, an instance of Unknown will be used to hold the record type and buffer.
-
idstools.unified2.
decode_record
(record_type, buf)[source]¶ Decodes a raw record into an object representing the record.
Parameters: - record_type – The type of record.
- buf – Buffer containing the raw record.
Returns: The decoded record as a
Event
,Packet
,ExtraData
orUnknown
if the record is of an unknown type.
-
idstools.unified2.
read_record
(fileobj)[source]¶ Reads a unified2 record from the provided file object.
Parameters: fileobj – The file like object to read from. Currently this object needs to support read, seek and tell. Returns: If a complete record is read a Record
will be returned, otherwise None will be returned.If some data is read, but not enough for a whole record, the location of the file object will be reset and a
EOFError
exception will be raised.