Skip to content

Troubleshooting

Troubleshooting Browsermon

This section provides some troubleshooting tips related to BrowserMon service.

Note: When troubleshooting, it’s essential to switch the log mode to DEBUG in the browsermon.conf file. This ensures that the service captures more granular details in the logs. After changing the log mode, restart the BrowserMon service to apply the new settings.

log_level=DEBUG
# options: INFO, DEBUG
# default: INFO
# description: Defines the logging level

Initial Checks

  1. Service Status

  2. Windows: Check if the service is running

    Start Menu > Services > BrowserMon

    Check the status of the service if it is running or not.

  3. Linux: Check if the service is running

    bash systemctl status borwsermon.service

  4. Permissions

BrowserMon service is meant to run with the system level privileges. Although the service will give an error if not run as an Administrator, make sure when you install it, you run it as Admin.

  1. Logs

  2. Review the service logs and the runtime-generated logs by BrowserMon. BrowserMon generates logs in the following location:

    • Windows: C:/browsermon/browsermon.log
    • Linux: /opt/browsermon/browsermon.log

  3. To see service logs that are logged by the operating system, do the following steps:

    • Windows: Typically found in "Event Viewer" > "Windows logs" > "Application" > "Browsermon"
    • Linux: Run the command journalctl -u browsermon.service

Known Issues

  1. 32-bit Json Support: Json mode is not supported on 32 bit systems.

  2. Watchdog Disconnection Issue: When disconnected from the watchdog server, the controller:

  3. Makes 10 reconnection attempts every 30 minutes
  4. Exits after 5 hours if reconnection fails
  5. Leaves reader processes as zombies
  6. Readers continue data collection but lose supervision features
  7. Configuration reload, automatic restart, and watchdog configuration fetch stop working

Resolution: powershell Stop-Process -Name "browsermon" -Force Restart-Service -Name "browsermon" This issue will be fixed in the upcoming release.

Common Issues and Solutions

  1. Service Won’t start

    If the service is not starting, review the logs of the service. If the service is crashing because of a runtime error, they will show in logs generated by the operating system. How to see logs? Please refer to Logs in the heading Initial Checks.

  2. Service is not behaving as expected

    If the service is not behaving as expected in the configuration, make sure there are no errors in the browsermon.conf file. If it contains any errors or typos, BrowserMon service will use default configurations, which are:

{
    "browser": "all",
    "mode": "scheduled",
    "schedule_window": "1m",
    "logdir": "C:\\browsermon\\history (Windows) or /opt/browsermon/history (Linux)",
    "logmode": "csv",
    "rotation": "1m",
    "backup_count": 5
}


  1. History is not being logged

    If the service is not logging history in the folder mentioned in the browsermon.conf file, make sure you supplied the correct folder and path, and indeed the folder exists where you wish the history to be written.

    Still, if history is not being logged, you can do the following checks:

    1. Make sure the "stop saving history" option in the browser is disabled.
    2. Make sure sync is on; sometimes when sync is off, history might not get saved.
    3. Restart the service.
    4. Restart the computer.
    5. Create a new profile and check if history of that file is being logged.

Advanced Troubleshooting

Windows

  • Use Event Viewer to find detailed error messages.
  • Execute sfc /scannow to check for and fix system file issues.
  • Check Windows updates and apply any pending updates.

Linux

  • Run dmesg | grep browser-monitor to check for kernel or hardware-related errors.
  • Check for and apply updates using your distribution’s package manager (e.g., apt, yum, etc.).
  • Use strace to monitor system calls made by the service for advanced debugging.

Using the Troubleshooter

Download the BrowserMon troubleshooter from the website, which will collect all logs and relevant information and save it into a file named browsermon_archive.zip, which you can then transfer to the team that will help you fix the issue.

How to run troubleshooter?

browsermon_ts troubleshoot -v

BrowserMon Troubleshooting CLI Options

The BrowserMon Troubleshooting CLI provides several options to customize the execution of the troubleshooting process. These options allow you to specify the location to store logs, set the verbosity level, and choose specific information to fetch.

CLI Options

--logs-dir

  • Description: Specifies the directory where a copy of the logs should be stored.
  • Default:
  • On Windows: C:\browsermon\logs
  • On other platforms: /opt/browsermon/logs
  • Usage: --logs-dir <path>
  • Example: --logs-dir C:\custom\log\directory

--service-dir

  • Description: Specifies the directory where the BrowserMon service files are located. This is useful when you have installed the service in a custom directory.
  • Default:
  • On Windows: C:\Program Files\BrowserMon
  • On other platforms: /opt/browsermon
  • Usage: --service-dir
  • Example: --service-dir C:\custom\service\directory

-vv (Verbose Mode)

  • Description: Collects all information, including procmon details, for a comprehensive troubleshooting report.
  • Usage: -vv
  • Example: browsermon_ts troubleshoot -vv

-v (Standard Mode)

  • Description: Runs all troubleshooting functions except for procmon, providing a standard level of detail in the report.
  • Usage: -v
  • Example: browsermon_ts troubleshoot -v

--browser-version

  • Description: Fetches only the browser version information. When this flag is used, no other information is collected unless explicitly specified with other flags.
  • Usage: --browser-version
  • Example: browsermon_ts troubleshoot --browser-version

--system-info

  • Description: Fetches only the system information. This flag can be used alone or in conjunction with other flags to include additional information in the report.
  • Usage: --system-info
  • Example: browsermon-ts troubleshoot --system-info

--service-info

  • Description: Fetches only the service information. This option is useful for narrowing down the troubleshooting to service-specific issues.
  • Usage: --service-info
  • Example: browsermon_ts troubleshoot --service-info

Examples

  1. To generate a verbose troubleshooting report (wihtout procmon) and store logs in the default directory:

    browsermon_ts troubleshoot -v

  2. To generate complete verbose troubleshooting report (including prcomon) and store logs in the default directory

    browsermon_ts troubleshoot -vv

NOTE: For this to work you must place procmon executeable in the same location from where you run the browsermon_ts You can download the procmon from Microsofts official site.

The troubleshooter uses it to monitor real-time file system, Registry, and process/thread activity of BrowserMon process. This is only supported on Windows.

This will open a procmon window; minimize it and let it run in the background until the troubleshooter has collected all the information. Once the status is at 100%, you can exit the procmon window and the troubleshooter window. The troubleshooter will create a file named browsermon_archive.zip in the directory provided with command line option --logs-dir

Privacy Disclaimer

BrowserMon troubleshooter collects all relevant information that can help debug the issue. For transparency, here is the list of items BrowserMon troubleshooter will collect:

  1. browsermon.log
  2. Installed Browser’s version Information
  3. System Information (i.e., platform, platform version, architecture, IP address, MAC address, processor, and RAM)
  4. Service Information
  5. Troubleshooter will run procmon to fetch real-time event properties related to BrowserMon. For more information on what procmon monitors, please take a look at their site here.

Troubleshooting Watchdog

This section provides some troubleshooting tips related to Watchdog.

Initial Checks

  1. Check Port Status If you are experiencing issues with the Watchdog server, the first step is to check the status of the port Watchdog is listening on. Follow these steps to troubleshoot:

    • Use netstat to Check Port Status bash netstat -tuln | grep <watchdog_port>
    • Check if Watchdog is listening on the port and bind to 0.0.0.0

  2. Logs

    • Set the loglevel to DEBUG in conf file.

    • Review the Watchdog logs and the Gunicorn Server logs. Watchdog generates logs in the logdir configured in conf file:

      • Watchdog Logs
      • Gunicorn Logs

  3. Process Status

  4. Check the status of the process if it is running or not.

    • Check if the process is running bash ps aux | grep Watchdog
    • Get the details of the process if it's running bash ps -p <pid> -o pid,ppid,cmd,%cpu,%mem,etime

Docker Troubleshooting

  • Check Docker logs for issues: bash docker logs container-name

  • If you encounter issues with Kafka, exec into the Kafka-Connect container: bash docker exec -it kafka-connect /bin/bash

  • Check the status of the MongoDB sink connector: bash curl -X GET http://kafka-connect:8083/connectors/mongo-sink-connector/status

json { "name": "mongo-sink-connector", "connector": { "state": "RUNNING", "worker_id": "connect-worker-1" }, "tasks": [ { "id": 0, "state": "RUNNING", "worker_id": "connect-worker-1" } ], "type": "sink" }

Kafka and Elasticsearch troubleshoot

1. Checking Logs

View logs for a specific container:

docker logs <container-name>

Examples: - docker logs kafka-connect - docker logs elasticsearch

2. Verifying Kafka Connect

Inside the Kafka Connect container:

docker exec -it kafka-connect /bin/bash

Then check connector status:

curl -X GET http://kafka-connect:8083/connectors/mongo-sink-connector/status

A valid Mongo Sink Connector shows:

{
  "name": "mongo-sink-connector",
  "connector": {
    "state": "RUNNING",
    "worker_id": "connect-worker-1"
  },
  "tasks": [
    {
      "id": 0,
      "state": "RUNNING",
      "worker_id": "connect-worker-1"
    }
  ],
  "type": "sink"
}

3. Checking Elasticsearch

If Elasticsearch is running with HTTPS and basic auth:

curl -k -u elastic:BrowsermonElasticAdmin https://localhost:9200/_cluster/health
  • -k ignores self-signed certificate errors.
  • Adjust the user/password as you configured them during installation prompts.

4. Internet Access

Important For the functioning of the Elasticsearch-based URL classification, the following domains must be accessible from the network where your watchdog is deployed.

For ETI:

  • PhishTank : data.phishtank.com
  • URLHaus : urlhaus.abuse.ch

For UCS (if ucs_updates=true in watchdog.conf):

  • EUNOMATIX Cloud : ucs.eunomatix.com:8000

Common Issues and Solutions

Server Not Starting

  • Review your Watchdog Config. Ensure that you are using compatible version of Watchdog.
  • Things that can go wrong in the config file.
  • MODE: Set the mode to either 'local' or 'cloud' based on your deployment.
  • BMKEY: Provide the Base64-encoded license key.
  • AUTHCODE: Add the Base64-encoded authorization code.
  • BLACKLIST: Specify the blacklist configuration.
  • LOGDIR: Path to your log directory.
  • LOGLEVEL: Set the desired loglevel. Set it to DEBUG if the program doesn't work the intended way.
  • LIMIT: Enable the rate limiting.
  • RATE: Give the rate on which API is accessible.

  • ALLOWEDHOSTS: Lists of the Hosts that can access Watchdog.

  • WATCHDOG_PORT: Port on which watchdog will be run.

  • CERTFILE: Path to the SSL Certificate File
  • KEYFILE: Path to the SSL Key File
  • SSLCONFIG: SSL Config to genrate the cert files.

Server Not Behaving As Expected

  • Review your SSL certificate is valid and not expired.
  • Ensure you are not exceeding the max controllers limit.

Contacting Support

Download the Watchdog troubleshooter from the website, to collect all logs and relevant information and save it into a file named watchdog_archive.zip, which you can share with the team that will help you fix the issue.

  • Run the script in the same folder as the Watchdog binary or optionally use absolute paths for LOGDIR, CERTFILE and KEYFILE in the config file.
  • Make sure that your config contains absolute path of log and cert files.
  • To run the troubleshooter, run the following command.
./WatchdogTS /path/to/your/watchdog.conf /path/to/your/Watchdog_binary

Privacy Disclaimer

Watchdog troubleshooter collects all relevant information that can help debug the issue. Here is the list of items BrowserMon troubleshooter will collect:

  1. watchdog_*.log watchdog_gunicorn.log
  2. SSL Certificates
  3. System Information (i.e., platform, platform version, architecture, IP address, MAC address, processor, and RAM)
  4. Process Information
  5. Troubleshooter will run lsof to fetch real-time event properties related to Watchdog.

For further details, you can take a look at the troubleshooter script on Eunomatix’s GitHub.