In opscotch version 3, file access was introduced, along with the fileWatcher step trigger. File watching is a capability that allows you to request the agent to call a step when a file is changed—for example, when a log file is updated, you can receive the new lines as a step payload. As security is always a top priority, file access is strictly controlled in several ways.

How does this work?

Watching a file utilizes the bootstrap file access property, which already provides the necessary file security mechanisms.

To watch a file or files in a directory, add a file access record to the bootstrap with the READ and LIST permissions.

Bootstrap:

{
    ...
    "allowFileAccess": [
        {
            "id": "watchMyDirectory",
            "directoryOrFile": "/a/path",
            "LIST": true,
            "READ": true
        }
    ],
    ...
}

Next add a step with the fileWatcher trigger. The following properties are required:

• bootstrapFileId: the id of the bootstrap allowFileAccess record
• eventSplitter: A (escaped) regex pattern to use to split the file contents into records.
  • Take note of the splitAtEnd property; this can be used to change the behavior of event splitting. For example, if the start of the record is a date, set the pattern to match the date and splitAtEnd: false - now the file will be split such that the date is at the start of the record.

Workflow:

{
    ...
    "workflows" : [
        {
            ...
            "steps" : [
                {
                    ...
                    "trigger" : {
                        "fileWatcher" : {
                            "bootstrapFileId" : "watchMyDirectory",
                            "eventSplitter" : "\\n" 
                        }
                    },
                    "resultsProcessor" : {
                        "processors" : [
                            {
                                "script" : "console.log(context.getMessageBodyAsString());"
                            }
                        ]
                    }
                    ...
                }
            ]
            ...
        }
    ]
    ...
}

When the workflow is loaded, opscotch will place a watch on the files, and when a change is made to the files, the step's resultsProcessor will be called with the following payload structure:

{
   "log": {
     "file": {
       "path": "/path/to/file.txt"
     },
     "offset": 4130
   },
   "message": "this is a line from a file",
   "input": {
     "type": "log"
   },
   "host": {
     "name": "hostname",
     "ip": "[fe80:0:0:0:a6d7:feea:f601:902%wlp1s0, 192.168.0.27]"
   },
   "agent": {
     "type": "opscotch",
     "version": "3.0.0"
   },
   "ecs": {
     "version": "1.12"
   }
}

You can then use workflows to process the file contents as you like.