In addition to importing real-time syslog data, Fastvue Sophos Reporter can also import historical logs exported from Sophos UTM.
This is done by checking the Import Historical Logs option in your Source in Settings | Sources, and entering the folder path where your historical web filtering logs are stored.
Your historical web filtering logs can be downloaded from your Sophos UTM (Log Settings | View Log Files | Archived Log Files then select the Web filtering log subsystem) or you can set up a remote log archive to have them automatically pushed to a remote folder at midnight each night. See Never Miss Reporting Data With Sophos UTM’s Remote Log Archive.
When attempting to import historical logs, an error is displayed reporting that Fastvue Sophos Reporter cannot connect to the log folder.
Sophos Reporter runs as a service under the local system account by default on the server it's installed on, so is unable to authenticate to remote server's file shares. When it attempts to connect to a remote share to access historical logs it can only use the account the service runs as to access the share.
The issue could also be due to the incorrect folder being specified, a mapped network drive being used, or using unexpected filenames, formats and folder structures.
A) Do not use Mapped Network DrivesDo not use a mapped network drive such as X:\Logs\Sophos. Instead, specify the UNC path, such as \\servername\Logs\Sophos. Mapped network drives only exist under a 'user' context and the Fastvue Reporter service running under the local system account cannot access them.
B) Troubleshoot Authentication/Security Issues
Option 1 - Import from a Local Folder
The easiest way to work around authentication/security issues is to have your UTM copy the logs directly to a folder on the Sophos Reporter server's local storage so that Sophos Reporter service has direct local access to the logs, and then use the local path in the historical import option in Sophos Reporter. E.g. C:\Logs\Sophos
Option 3 - Add Read Permission for the Fastvue Server to the Log Folder
If your Sophos Reporter server and log server are both joined to the same domain, you can add permissions for the server itself to the logs folder which should grant access to the Sophos Reporter service.
You can do this when adding security permissions to your logs folder by clicking Object Types in the Select Users, Computers, Service Accounts or Groups dialog and ensuring Computers is checked, and then type the name of the Fastvue server with a $ symbol after it (e.g. REPORTSVR$). Add at least read permission to the share, and ensure on the Security tab in the log folder properties that there are filesystem permissions for the Fastvue server as well.
Option 3 - Create a new account for the Fastvue Reporter service
Create a domain system account for Sophos Reporter. Grant the system account access to the remote log folder and set the Sophos Reporter service to run as that domain user with full permissions to the Sophos Reporter data location (shown in Settings | Data Storage | Settings).
*Note: If Fastvue Sophos Reporter is reinstalled or updated, the service will be removed and re-added, and the user account that the service is running as will need to be updated again.
C) Use the expected filenames, formats and folder structure
Fastvue Sophos Reporter was designed to work with the logs exported automatically by the Sophos UTM, so it will be expecting the files to be similarly structured and named. Therefore, ensure the log files are not located in sub-folders and are using the expected format and filenames.
Fastvue Reporter will only import the Web Filtering logs from Sophos UTM. These logs may be in the form of the Web Filter module's daily logs (http -2017-05-02.log.gz), or within the tarball containing logs from all modules for a day (logfile-2017-06-02.tgz), so you can have your logs in either of these forms and Fastvue Reporter will automatically work out how to import them.
Just ensure is that the log's date is in the filename in either yyyy -MM-dd-module or module_yyyyMMdd syntax (where yyyy is Year, MM is Month, and dd is Day).
The reason why the date must be in the filename is so that Fastvue Reporter can import only logs from dates before the Source was added in Fastvue Reporter, to avoid duplication of data.
Data is supported in .gz, .tgz, .tar.gz, and .tar. so it's ok to have the logs compressed in these formats.
Making the change that works for your environment you should now be able to Import historical log files.
As always, if you need additional assistance, please get in touch!