Monitoring system metrics, processes, and file size using node exporter
We all know that node_exporter is one of the widely used exporters for exporting system metrics, but only a few of us are aware that we can even use node-exporter to monitor systemd processes and file sizes as well. If you’re not aware of it, this article is for you.
Here in this article, we will guide you step by step to monitor
- System metrics
2. File size
3. Processes
If you don’t have promotheus installed in your system, use this guide to install it
- Exporting system metrics with node-exporter :
If you want to monitor your server's system metrics like CPU, memory, disk usage, etc… Then node-exporter is best because it can collect all the system metrics and expose them on port 9100, if you want to collect system metrics using node-exporter then you need to install the node-exporter in all the servers whichever you want to monitor.
Below are the installation and configuration steps.
- Download the latest version of node-exporter from the Prometheus release page using the below command :
cd /tmp
curl -LO https://github.com/prometheus/node_exporter/releases/download/v1.4.0/node_exporter-1.4.0.linux-amd64.tar.gz- Extract the .gz file using tar :
tar -xvf node_exporter-1.4.0.linux-amd64.tar.gz- Move the node exporter binary to /usr/local/bin directory :
sudo mv node_exporter-1.4.0.linux-amd64/node_exporter /usr/local/bin/- Create a node_exporter user :
sudo useradd -rs /bin/false node_exporter- Create a node-exporter systemd service
sudo vim /etc/systemd/system/node_exporter.serviceNote: in vim use “i” for insert mode, “escape + wq!” To save and exit
- Add the below content in the file (/etc/systemd/system/node_exporter.service)
[Unit]Description=Node ExporterAfter=network.target[Service]User=node_exporterGroup=node_exporterType=simpleExecStart=/usr/local/bin/node_exporter[Install]WantedBy=multi-user.target
- Reload system-daemon and start node_exporter
sudo systemctl daemon-reloadsudo systemctl start node_exporter
- Check the status of the node exporter
sudo systemctl status node_exporter- Now you can able to see the metrics on port 9100, open the browser to see the metrics
SERVER_IP:9100/metricsSo now we are adding this server as a target in the Prometheus to query metrics and create alerts
Note: This configuration should be done in the Prometheus server
We are not covering Prometheus installation and configuration in this article, you can check that here 👇
Log in to the Prometheus server and follow the below steps
- Open the configuration file using vim
sudo vim /etc/prometheus/prometheus.yml- Add the node exporter installed server as a target under scrape configs
- job_name: 'node_exporter_metrics'
scrape_interval: 5s
static_configs:
- targets: ['SERVER_IP:9100']- Restart Prometheus
sudo systemctl restart prometheus- Visit the Prometheus dashboard on PROMETHEUS-SERVER-IP:9090
Now you can able to see the node-exporter-installed server under targets in the Prometheus dashboard
Also, you can use the Prometheus expression section in the dashboard to query for node-related metrics. Following are a few key node metrics you can use to find its statistics.
node_memory_MemFree_bytes
node_cpu_seconds_total
node_filesystem_avail_bytes
rate(node_cpu_seconds_total{mode=”system”}[1m])
rate(node_network_receive_bytes_total[1m])2. Configuring node_exporter to get “processes” metrics
As you can see by default node_exporter only show system metrics. To get the processes metrics we need to use one of the node_exporter collectors called “systemd” but it is limited to systemd processes only. If you want to monitor any binary processes then you need to use process-exporter
- Enable systemd collector of node_exporter
Edit the node exporter service file Add the systemd as shown below
- Edit the node-exporter service file
sudo vim /etc/systemd/system/node_exporter.service- Modify the service file ExecStart and append — collector.systemd like below
[Unit]Description=Node ExporterAfter=network.target[Service]User=node_exporterGroup=node_exporterType=simpleExecStart=/usr/local/bin/node_exporter — — collector.systemd[Install]WantedBy=multi-user.target
- restart the node_exporter
systemctl restart node_exporterAlso, you can use the Prometheus expression in the Prometheus dashboard to query for process-related metrics.
- Here are a few popular node exporter systemd metrics
node_systemd_unit_state- You can use the below query in Prometheus to check particular process is running or not on a particular server
node_systemd_unit_state{instance=”server-name”, name=”httpd”, state=”active”} == 0You can also set up alerts based on these queries.
3. In this section, I’ll cover how to monitor file size using node-exporter
Yes, You can even monitor file size using node-exporter, There’s another node-exporter collector called “- — textfile” which exposes statistics read from the local disk.
Follow these steps to enable file size monitoring using textfile collector
- Create a directory ( you can create one anywhere )
mkdir /usr/local/node_exporter- Create a cronjob for every minute to get the file size and copy it to “/usr/local/node_exporter/” directory as a “.prom” file (as shown in the below steps).
- run the below command to edit cronjobs
crontab -e- Add the below content and save it, in my case I’m monitoring the test.txt file located in /var/test, make sure to replace the file path as per your requirement.
* * * * * echo $(ls -s /var/test/test.txt | awk ‘{ print “gearman_sqlite_db_file_size “ $1 }’) > /usr/local/node_exporter/test-file-size.prom- To check the file size using node-exporter, we need to use the “ collector.textfile.directory” collector of node-exporter.
- run the below command to edit the service file using vim (Note: usually service files are located in “/etc/systemd/system/ “ )
vim /etc/systemd/system/node_exporter.service- Add “- -collector.textfile.directory” , /usr/local/node_exporter directory ( above created directory to monitor) at the end of the “ExecStart” section of node_exporter. service file and save the changes. (as shown below)
[Unit]Description=Node ExporterWants=network-online.targetAfter=network-online.target[Service]User=node_exporterGroup=node_exporterType=simpleExecStart=/usr/local/bin/node_exporter — collector.textfile.directory /usr/local/node_exporter[Install]WantedBy=multi-user.target
- Restart the node_exporter
systemctl restart node_exporterNow you can able to see the file size metrics.
