RegEx FlexConnector Not Parsing.

Below I have posted a small set of sample logs, the code for my flexagent parser, and the entry I have in the agents.properties file for a custom syslog parser.  Is there anything that stands out as to why the data is not being parsed?  If I look in the agents.log file, I see the following error: 

[2024-05-30 15:03:42,809][WARN ][com.arcsight.agent.parsers.operation.regexTokenOperation] [getResult]No match between string [May 30 12] and regex [(\S+) \S+ (?:login|sshd|httpd)]

Parents
  • 0

    This is the sample of the logs and the Flexagent parser below

    # <185>May 23 2024 11:51:15 %FTD-1-430003: EventPriority: Low, DeviceUUID: dd4d1b20-e37a-11ee-a052-870306bb8e79, InstanceID: 1, FirstPacketSecond: 2024-05-23T11:51:15Z, ConnectionID: 24226, AccessControlRuleAction: Allow, SrcIP: 172.16.21.5, DstIP: 162.232.201.14, SrcPort: 63451, DstPort: 443, Protocol: tcp, IngressInterface: Inside, EgressInterface: Outside, IngressZone: INSIDE, EgressZone: OUTSIDE, IngressVRF: Global, EgressVRF: Global, ACPolicy: AI-PDP-1150-DC-01 Access policy, AccessControlRuleName: New-Rule-#2-ALLOW, Prefilter Policy: Default Prefilter Policy, ConnectionDuration: 0, InitiatorPackets: 1, ResponderPackets: 1, InitiatorBytes: 59, ResponderBytes: 70, NAPPolicy: Balanced Security and Connectivity


    regex=.*<\\d+>([\\w\\d\\s:]+)\\s\\s\\s.*AccessControlRuleAction:\\s([^,]+),\\sSrcIP:\\s([^,]+),\\sDstIP:\\s([^,]+),\\sSrcPort:\\s([^,]+),\\sDstPort:\\s([^,]+),\\sProtocol:\\s([^,]+),\\sIngressInterface:\\s([^,]+),\\sEgressInterface:\\s([^,]+),\\s.*ACPolicy:\\s([^,]+),\\sAccessControlRuleName:\\s([^,]+),\\s.*InitiatorBytes:\\s([^,]+),\\sResponderBytes:\\s([^,]+),.*


    token.count=13


    token[0].name=timestampp
    token[0].type=String

    token[1].name=deviceactions
    token[1].type=String

    token[2].name=sourceaddresses
    token[2].type=IPAddress

    token[3].name=destinationaddresss
    token[3].type=IPAddress

    token[4].name=sourceports
    token[4].type=Integer

    token[5].name=destinationports
    token[5].type=Integer

    token[6].name=protocolss
    token[6].type=String

    token[7].name=deviceoutbounddirections
    token[7].type=String

    token[8].name=deviceinbounddirections
    token[8].type=String

    token[9].name=Policy
    token[9].type=String

    token[10].name=Rule
    token[10].type=String

    token[11].name=bytesinn
    token[11].type=Integer

    token[12].name=bytesoutt
    token[12].type=Integer

    additionaldata.enabled=true

    event.name=__stringConstant("Firewall Events")
    event.timestamp=timestampp
    event.deviceAction=deviceactions
    event.sourceAddress=sourceaddresses
    event.destinationAddress=destinationaddresss
    event.sourcePort=sourceports
    event.destinationPort=destinationports
    event.transportProtocol=protocolss
    event.deviceInboundInterface=deviceinbounddirections
    event.deviceOutboundInterface=deviceoutbounddirections
    event.deviceCustomString1=Policy
    event.deviceCustomString2=Rule
    event.bytesIn=bytesinn
    event.bytesOut=bytesoutt



  • 0   in reply to 

    The main problem is that the syslog header is not RFC compliant. (hostname/IP is missing). I would take the combined approach of a simple regex parser and a key-value extra processor for all the fields after # <185>May 23 2024 11:51:15 %FTD-1-430003:  

    To capture the hostname use the variable _SYSLOG_SENDER or _SYSLOG_SOURCE_ADDR

    This makes the process of parsing and mapping very easy.

Reply
  • 0   in reply to 

    The main problem is that the syslog header is not RFC compliant. (hostname/IP is missing). I would take the combined approach of a simple regex parser and a key-value extra processor for all the fields after # <185>May 23 2024 11:51:15 %FTD-1-430003:  

    To capture the hostname use the variable _SYSLOG_SENDER or _SYSLOG_SOURCE_ADDR

    This makes the process of parsing and mapping very easy.

Children
No Data