chenjl/canal
1
0
Fork 0
mirror of https://github.com/alibaba/canal.git synced 2026-11-04 00:30:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,692 Commits 13 Branches 38 Tags
5607444bad100a9b7aef0bbdb7e4b2152165a371
Commit Graph

1692 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
jianghang.loujh
5607444bad merge from github and rebase 2025-07-28 14:38:10 +08:00
jianghang.loujh
3a7223e460 merge from github 2025-07-28 14:37:47 +08:00
Canlin Guo
553438e227 CI: add a workflow for CI (#5483) 
* Create SECURITY.md

* CI: Create maven.yml

* Update maven.yml

* Update maven.yml

* fix bugs of fastsql schema test
 2025-07-28 14:33:52 +08:00
Canlin Guo
a3ea3d32d9 Create SECURITY.md (#5472) 2025-06-27 19:05:52 +08:00
Sunxien®
50a570290a fix(canal/client-adapter): 修复adapter插件重试错误缺陷 (#5427) 
* fix(canal/prometheus): 修复延迟指标的缺陷

-【缺陷描述】
   Admin控制台修改canal.properties配置后，Server自动进行重启，在Grafan上观测到同步实例的PUT/GET/ACK延迟居高不下。

-【原因定位】
   源库心跳正常触发，MemoryStoreWithBuffer正常推进位点，profiling 正常统计，但是延迟指标依然是越来越来越高。Prometheus调用collect接口采集到的exec time时间始终是固定不变的。经debug排查到StoreCollector采集器内存hold的StoreMetricsHolder与CanalInstance实例中的引用已经不同啦，CanalIntance重启时已经被重建过一份新的实例，但是StoreMetricsHolder却没有保存到内存Hold中。原因是Map.putIfAbsent调用引起。

-【修复效果】
- 修复后，重启Server心跳正常推进，延迟瞬间降下来。梳理其他Collector代码都是调用Map.put，只有这里使用putIfAbsent可能是粗心导致吧。修复效果

* fix(canal/client-adapter): 修复adapter插件重试错误缺陷

-【缺陷描述】
  adapter运行期间，针对server或者Instance进行过一次重启或者断网测试，发现adapter通过client拉取数据报错以后持续打印错误日志，永远不得恢复。虽然adapter processor 针对错误有重试逻辑，但是client的tcp连接已经发生中断，且无法恢复。

-【问题定位】
  adapter processor 针对错误的处理比较粗，需要细分是从上游get message报错，还是下游sync data报错？
  如果是 get message报错，特别是tcp 断开，需要跳出重试，重新尝试重连。如果是下游 sync data 报错，再考虑重试写入。下游插件开发者自行保证连接重连机制、写入幂等。框架只提供重试。

- 说明：顺带更新一下 .gitignore 文件
 2025-05-16 15:12:42 +08:00
Sunxien®
6b7ddbbe92 fix(canal/prometheus): 修复延迟指标的缺陷 (#5426) 
-【缺陷描述】
   Admin控制台修改canal.properties配置后，Server自动进行重启，在Grafan上观测到同步实例的PUT/GET/ACK延迟居高不下。

-【原因定位】
   源库心跳正常触发，MemoryStoreWithBuffer正常推进位点，profiling 正常统计，但是延迟指标依然是越来越来越高。Prometheus调用collect接口采集到的exec time时间始终是固定不变的。经debug排查到StoreCollector采集器内存hold的StoreMetricsHolder与CanalInstance实例中的引用已经不同啦，CanalIntance重启时已经被重建过一份新的实例，但是StoreMetricsHolder却没有保存到内存Hold中。原因是Map.putIfAbsent调用引起。

-【修复效果】
- 修复后，重启Server心跳正常推进，延迟瞬间降下来。梳理其他Collector代码都是调用Map.put，只有这里使用putIfAbsent可能是粗心导致吧。修复效果
 2025-05-16 15:09:22 +08:00
kkz-01
2b53164253 解决8.0.30+使用cahcing_sha2_password时Fastauth报错问题 (#5406) 
Co-authored-by: zhangkai <zhangkai02@xgd.com>
 2025-05-16 15:06:36 +08:00
Sunxien®
e101287c25 fix(canal/startup.sh): 修复生产环境ZK启用sasl安全认证 (#5438) 
* fix(canal/startup.sh): 修复生产环境ZK启用sasl安全认证

- 如果生产环境zk启用sasl安全认证，canal-server/canal-adapter启动无法连接zk进行读写操作
- 默认支持用户将 jaas.conf 放置在 ./conf/ 目录下，启动脚本增加-Djava.security.auth.login.config 启动参数即可

* fix(canal/parent): upgrade h2 version

- runtime error: Value too long for column "CHARACTER VARYING" #5086

- Ref: https://github.com/alibaba/canal/issues/5086
 2025-05-16 15:04:02 +08:00
elegance code
b153e61f0d fix issue #5401 (#5402) 2025-05-16 15:03:01 +08:00
hygugb
9bd4357f37 fix: tsdb存储元数据时支持大对象（大于64M） (#5393) 2025-05-16 15:00:19 +08:00
jianghang.loujh
dad209feb8 fixed issue #5453 , skipping Query_log_event by unknown status vars 2025-05-16 14:57:54 +08:00
jianghang.loujh
e21e98aa83 fixed CanalAdmin instanceLog and polardbx columnar test 2025-03-19 15:51:27 +08:00
nakeiven
c998b3f100 Feature/adapter postgresql 20250124 (#5390) 
* adapter tcp模式mysql数据全量同步至postgresql时 报错
fix issue 2146

* fix issue 1980

* feat 增加 当dbType 为 postgresql时，返回双引号(避免表名为postgresql的关键字时，sql查询报错)

---------

Co-authored-by: guodong.zhang <zgd8988@126.com>
 2025-01-24 17:14:41 +08:00
jianghang.loujh
c15129b0ac [maven-release-plugin] prepare for next development iteration 2025-01-16 16:07:08 +08:00
jianghang.loujh
0741ccda1e [maven-release-plugin] prepare release canal-1.1.8 canal-1.1.8 2025-01-16 16:07:00 +08:00
jianghang.loujh
02fffc9e74 compatiable polardb-x 2025-01-13 22:07:12 +08:00
jianghang.loujh
5ca4c39313 compatiable percona & polardb-x 2025-01-13 22:02:09 +08:00
jianghang.loujh
3162d7093d fixed polardb-x variables(flashback & gpp) 2025-01-13 21:35:33 +08:00
jianghang.loujh
f350fbdc0a fixed mysql 8.0.30+ , support new variable 2025-01-08 10:10:33 +08:00
jianghang.loujh
2d0dcf0e5d fixed issue #5301, fixed caching_sha2_password full auth & client SHA2 2024-12-24 17:51:31 +08:00
jianghang.loujh
515c05c873 fixed issue #5301, fixed caching_sha2_password full auth 2024-12-24 16:39:21 +08:00
jianghang.loujh
7ee551cbf7 fixed issue #5301, fixed caching_sha2_password full auth 2024-12-24 16:33:41 +08:00
jianghang.loujh
923675aa49 fixed issue #5334 , adjust canal_manager default schema 2024-12-17 16:06:57 +08:00
jianghang.loujh
ebb541001c fixed issue #5337 , remove docker privileged 2024-12-17 15:52:01 +08:00
Chongzhi Deng
44ed76ebb8 fix typo (#5299) 
Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
 2024-12-17 15:32:03 +08:00
jianghang.loujh
fe836f7515 fixed issue #5336 #5332 , etl bugfix 2024-12-17 15:29:51 +08:00
quanllong
b30544c750 bugfix: 兼容MariaDB GTID 为空的情况 (#5352) 2024-12-17 15:09:41 +08:00
lol
66483ef83f Feature/helm charts (#5285) 
* canal-admin canal-server k8s helm chart

* 拼写错误
 2024-10-14 16:48:05 +08:00
jianghang.loujh
b391188baf code format & fine-tune 2024-10-14 16:44:59 +08:00
lol
2a7a507987 fixed health.sh failed (#5281) 
fixed health.sh failed
 2024-10-14 16:44:15 +08:00
lanicc
f53749bfc5 fixed issue #5289 (#5290) 
Co-authored-by: lanicc <liaonan@souche.com>
 2024-10-14 16:18:21 +08:00
jianghang.loujh
80ca4366db fixed #5270 , add CanalMQStarter stop latch 2024-09-13 18:04:56 +08:00
lwd-coding
be0d945d7d 当zk出现session time out可能会导致instance出现多个消费者，从而出现乱序 (#5270) 2024-09-13 17:02:19 +08:00
jianghang.loujh
5a968b3715 fixed typo 2024-09-11 15:05:36 +08:00
jianghang.loujh
1e5b8a2021 fixed schema blackFilter & mysql 8.4 2024-08-29 13:07:26 +08:00
hbzheng8
ff82fd6532 直接close socket，解决TLS half-close policy报错问题 (#5240) 2024-08-29 12:47:50 +08:00
Yifan Xia
b060f46ccf Update AbstractRequest.java (#5208) 2024-08-29 12:47:09 +08:00
TK337
7e31b40531 support mysql 8.4 (#5231) 2024-08-29 12:43:06 +08:00
dingxiaobo
b2b3223d1d fix mysql 5.1 negotiate NPE. (#5255) 2024-08-29 12:42:31 +08:00
jianghang.loujh
fea4d8210a fixed pull #5191 , reformat 2024-07-16 15:02:10 +08:00
JZ
fb3d2c6811 add compressionType,enableChunking for pulsar client , upgrade dependencies for security (#5191) 
* add support for listener name which is described in https://pulsar.apache.org/docs/next/concepts-multiple-advertised-listeners/

* upgrade for CVE-2023-20883  CVE-2022-1471  CVE-2023-20860

* add   enableChunking

* downgrade  pulsar client version

* fix

* 增加压缩算法
 2024-07-16 14:59:29 +08:00
jianghang.loujh
3439c48948 fixed pull #5175 , reformat 2024-07-16 14:57:53 +08:00
iamssx
2b8a309ec7 fixed Rocketmq can not batch send messages when producer set namespace (#5202) 2024-07-16 14:51:56 +08:00
Yazid Lee
9373429015 fix: resource loading failure caused by incorrect parameter order in StringUtils.startsWithIgnoreCase (#5173) 2024-06-26 15:53:24 +08:00
lwd-coding
9493ff733d fix: 同个instance配置多个group避免共享CanalMsgConsumer (#5175) 2024-06-26 15:49:48 +08:00
Tass
31530dc9af fix(adapter): add args into adapter/restart.sh (#5149) 
Signed-off-by: Tass <tasszz2k@gmail.com>
 2024-06-26 15:34:30 +08:00
jianghang.loujh
b0f3ff8dcc fixed pull #5147, reformat & add more configs 2024-06-26 15:33:14 +08:00
dingxiaobo
5c61f2fb56 Feat, support SSL(TLS), improve SHA2 password. (#5147) 
* feat #4489, mysql SSL

* feat #4489, mysql SHA2 and SSL

* merge master, fix conflict.

* improve SHA2 auth plugin, fix SSL status print.

* SSL key store, use url rather than path, keep same as JDBC driver.
 2024-06-26 15:06:16 +08:00
jianghang.loujh
7bb619cf12 fixed #5154,avoid spring RCE 2024-06-06 15:29:17 +08:00
lulu2panpan
d66a15fe35 migrate sql parser from druid to polardbx-parser (#5097) 
* migrate parser from druid to polardbx-parser

* migrate parser from druid to polardbx-parser(with version 5.4.19)

* fix bug : full ddl sqls should not assign to extra field but data field

* fix bug : full ddl sqls should not assign to extra field but data field

---------

Co-authored-by: agapple <jianghang.loujh@alibaba-inc.com>
canal-1.1.8-alpha-2 		2024-05-25 22:05:06 +08:00