Promethues 监控tomcat 主要用的模块

Promethus 负载抓取/存储指标信息、并提供查询功能
grafana 数据可视化
JMX exporter 提供JMX中JVM相关的metrics

1、利用JMX exporter，在Java进程内启动一个小型的Http server 2、配置Prometheus抓取那个Http server提供的metrics。3、配置Grafana连接Prometheus，配置Dashboard。

一、下载一个tomcat,下载jmx_exporter

1、获取jmx_exporter有2种方法，自己编译获取jar包，获取现成的jar包

wget https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.12.0/jmx_prometheus_javaagent-0.12.0.jar

或者编译 git clone https://github.com/prometheus/jmx_exporter cd jmx_exporter mvn package
2、安装tomcat

wget http://mirrors.tuna.tsinghua.edu.cn/apache/tomcat/tomcat-8/v8.5.51/bin/apache-tomcat-8.5.51.tar.gz tar zxvf apache-tomcat-8.5.51.tar.gz mv apache-tomcat-8.5.51 /usr/localt/tomcat_test
3、配置文件下载

wget https://github.com/prometheus/jmx_exporter/blob/master/example_configs/tomcat.yml

tomcat.yml的内容如下

---
lowercaseOutputLabelNames: true
lowercaseOutputName: true
rules:
- pattern: 'Catalina<type=GlobalRequestProcessor, name=\"(\w+-\w+)-(\d+)\"><>(\w+):'
  name: tomcat_$3_total
  labels:
    port: "$2"
    protocol: "$1"
  help: Tomcat global $3
  type: COUNTER
- pattern: 'Catalina<j2eeType=Servlet, WebModule=//([-a-zA-Z0-9+&@#/%?=~_|!:.,;]*[-a-zA-Z0-9+&@#/%=~_|]), name=([-a-zA-Z0-9+/$%~_-|!.]*), J2EEApplication=none, J2EEServer=none><>(requestCount|maxTime|processingTime|errorCount):'
  name: tomcat_servlet_$3_total
  labels:
    module: "$1"
    servlet: "$2"
  help: Tomcat servlet $3 total
  type: COUNTER
- pattern: 'Catalina<type=ThreadPool, name="(\w+-\w+)-(\d+)"><>(currentThreadCount|currentThreadsBusy|keepAliveCount|pollerThreadCount|connectionCount):'
  name: tomcat_threadpool_$3
  labels:
    port: "$2"
    protocol: "$1"
  help: Tomcat threadpool $3
  type: GAUGE
- pattern: 'Catalina<type=Manager, host=([-a-zA-Z0-9+&@#/%?=~_|!:.,;]*[-a-zA-Z0-9+&@#/%=~_|]), context=([-a-zA-Z0-9+/$%~_-|!.]*)><>(processingTime|sessionCounter|rejectedSessions|expiredSessions):'
  name: tomcat_session_$3_total
  labels:
    context: "$2"
    host: "$1"
  help: Tomcat session $3 total
  type: COUNTER

4、收集数据 4-1、Tomcat收集数据

mkdir /usr/local/tomcat_test/jmx/ cp jmx_prometheus_javaagent-0.12.0.jar /usr/local/tomcat_test/jmx/ cp tomcat.yml /usr/local/tomcat_test/jmx/

修改配置文件vim /usr/local/tomcat_test/bin/catalina.sh JAVA_OPTS="-javaagent:/usr/local/tomcat_test/jmx/jmx_prometheus_javaagent-0.12.0.jar=39081:/usr/local/tomcat_test/jmx/tomcat.yml"

4-2、JAR包运行（案例，关键文件jmx_prometheus_javaagent-0.3.0.jar和tomcat.yml）

java -javaagent:./jmx_prometheus_javaagent-0.3.0.jar=9151:tomcat.yaml -jar yourJar.jar

5、测试是否收集到数据 curl -s http://localhost:39081/ | more

[root@Prometheus promethus]# curl -s http://localhost:39081/ | more

HELP jmx_exporter_build_info A metric with a constant '1' value labeled with the version of the JMX exporter.

TYPE jmx_exporter_build_info gauge

jmx_exporter_build_info{version="0.12.0",name="jmx_prometheus_javaagent",} 1.0

HELP jmx_config_reload_success_total Number of times configuration have successfully been reloaded.

TYPE jmx_config_reload_success_total counter

jmx_config_reload_success_total 0.0

HELP jmx_config_reload_failure_total Number of times configuration have failed to be reloaded.

TYPE jmx_config_reload_failure_total counter

jmx_config_reload_failure_total 0.0

HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.

TYPE process_cpu_seconds_total counter

process_cpu_seconds_total 3.99

HELP process_start_time_seconds Start time of the process since unix epoch in seconds.

TYPE process_start_time_seconds gauge

process_start_time_seconds 1.582290765496E9

HELP process_open_fds Number of open file descriptors.

TYPE process_open_fds gauge

process_open_fds 74.0

二、配置promethus 采集数据

1、文件引用 (本文采用) /usr/local/prometheus/prometheus.yml
- job_name: 'tomcat' file_sd_configs:
  - files: ['/usr/local/prometheus/conf/tomcat.yml'] refresh_interval: 180s

/usr/local/prometheus/conf/tomcat.yml/tomcat.yml

- targets:
  - 114.67.116.119:39081
  labels:
    idc: test_idc
    service: tomcat_test

2、直接配置

scrape_configs: - job_name: 'java' static_configs: - targets: ['114.67.116.119:39081']
3、重载配置文件

kill -hup ps -ef |grep prometheus|grep -v grep|awk '{print $2}' 或者 kill -9 PID /usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml
4、查看promethus界面，tomcat的监控是否发现