From c5a8d64169c9bcf572d1effd57cbec2fa1d51547 Mon Sep 17 00:00:00 2001 From: lcybo Date: Wed, 15 Aug 2018 23:46:15 +0800 Subject: [PATCH] Updated Canal prometheus (markdown) --- Canal-prometheus.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/Canal-prometheus.md b/Canal-prometheus.md index bd9f755..8662041 100644 --- a/Canal-prometheus.md +++ b/Canal-prometheus.md @@ -61,6 +61,7 @@ clamp_max(rate(canal_instance_sink_blocking_time{destination="example"}[2m]), 10 _简单说明一下range-vector,通俗来说表达式会用时间点前range-vector period内的所有samples参与运算。_ _range-vector值如果太小，图会碎片化；反之，实时性会比较差，不能及时反应变化。请结合scrape_interval设定合理的值。_ + * Dump线程空闲比 ``` clamp_max(rate(canal_instance_publish_blocking_time{destination="example"}[2m]), 1000) / 10 @@ -71,6 +72,7 @@ clamp_max(rate(canal_instance_publish_blocking_time{destination="example"}[2m]), **2. Sinking idle ratio较低，那么server端parser是性能瓶颈，可参考[Performance](https://github.com/alibaba/canal/wiki/Performance)进行tuning.** + * Delay(seconds) ``` canal_instance_traffic_delay / 1000 @@ -81,18 +83,21 @@ canal_instance_traffic_delay / 1000 **2. Limitations: 如果store满了(恰好store中的数据包含最新的position，且MySQL binlog停止更新)，且client暂停消费，那么delay会不断增长。当然，满足这些条件的概率极低。** + * Binlog接收带宽(KB/s) ``` rate(canal_instance_received_binlog_bytes{destination="example"}[2m]) / 1024 ``` **'Sink线程空闲比'与'Dump线程空闲比'都很低，delay还是很高的情况，请查看binlog接收速率是否符合预期。** + * TPS(Binlog events) ``` rate(canal_instance_store_consume_seq{destination="example"}[2m]) ``` **Canal instance client消费TPS(以binlog event为单位)，即canal instance最终的整体性能。** + * Client消费带宽(KB/s) ``` rate(canal_instance_client_bytes{destination="example"}[2m]) / 1024 @@ -101,36 +106,42 @@ rate(canal_instance_client_bytes{destination="example"}[2m]) / 1024 **注意：client在binlog无更新时busy poll，所占用的带宽也是颇为可观的，如上图。** + * TPS(MySQL transaction) ``` rate(canal_instance_transactions{destination="example"}[2m]) ``` **Canal instance处理transaction的TPS，以TRANSACTION_END事件为基准。** + * TPS(Table row) ``` rate(canal_instance_rows_counter{destination="example"}[2m]) ``` **Canal instance接受binlog中，数据库表变更行数的TPS.** + * DML model indicator ``` rate(canal_instance_rows_counter{destination="example"}[15s]) / rate(canal_instance_row_events{destination="example"}[15s]) ``` **RowData events包含行数。OLTP下，RowData大体上仅包含1行变更；批量DML下，该指标会上行。** + * RPS(Client request) ``` rate(canal_instance_client_packets{destination="example"}[2m]) ``` **Netty server处理client requests吞吐量。** + * QPS(空包) ``` rate(canal_instance_client_empty_batches{destination="example"}[2m]) ``` **client get返回每秒空包量。如果正常traffic下，该值很大，考虑使用connector的timeout机制，节省资源。** + * Client latency _experimental_ @@ -139,18 +150,21 @@ canal_instance_client_request_latency_bucket ``` **Histogram, client请求响应情况。关于[histogram](https://prometheus.io/docs/concepts/metric_types/#histogram).** + * Event store占用 ``` canal_instance_store_produce_seq - canal_instance_store_consume_seq ``` **Event store内未ack的events数量，实时性受scrape_interval影响。** + * Event store memory占用(KB, 仅memory mode) ``` (canal_instance_store_produce_mem - canal_instance_store_consume_mem) / 1024 ``` **Event store内未ack的events所占用内存大小，实时性受scrape_interval影响。** + * 状态信息 ``` canal_instance