chenjl/canal
1
0
Fork 0
mirror of https://github.com/alibaba/canal.git synced 2026-11-04 00:30:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,725 Commits 13 Branches 38 Tags
master
Commit Graph

192 Commits

Author SHA1 Message Date
jianghang.loujh
cf97b2ae31 fix: mitigate SnakeYAML deserialization RCE and enforce default password change 
1. Use SafeConstructor in ApplicationConfigMonitor to block arbitrary
   object instantiation via YAML type tags (e.g. !!javax.script.ScriptEngineManager).
2. Detect default admin password (admin/123456) on login and enforce
   password change before allowing any other API operations (code 50015).

🤖 Generated with [Qoder][https://qoder.com]
 2026-04-29 18:07:02 +08:00
jianghang.loujh
4ded2869c9 merge code 2026-01-28 12:06:35 +08:00
sunxien
dcea8afc93 refactor(canal): 升级JVM参数 (#5544) 
- 重点升级JVM参数

- 重构SessionHandler代码, 处理错误请求会响应两次包；
 2026-01-28 11:20:19 +08:00
jianghang.loujh
d18bde71c6 merge and support polardb-x recycle_bin binlog format 2025-12-25 11:47:36 +08:00
sunxien
5a5e60d371 feat(canal/adapter): adapter支持prometheus暴露jvm指标 (#5549) 
- adapter 默认提供prometheus暴露JVM指标

- 指定 maven-antrun-plugin 未指定版本号，默认是 3.0.0，但是配置中使用 <tasks> 标签，编译报错。
   - 降级版本号，显式指定 1.8 版本
   - 或者把配置中的 <tasks> 标签改成 <target>
 2025-12-25 11:41:08 +08:00
jianghang.loujh
556d998db1 adjust preferIPv4Stack=false, support ipv6 2025-08-05 16:42:00 +08:00
Sunxien®
50a570290a fix(canal/client-adapter): 修复adapter插件重试错误缺陷 (#5427) 
* fix(canal/prometheus): 修复延迟指标的缺陷

-【缺陷描述】
   Admin控制台修改canal.properties配置后，Server自动进行重启，在Grafan上观测到同步实例的PUT/GET/ACK延迟居高不下。

-【原因定位】
   源库心跳正常触发，MemoryStoreWithBuffer正常推进位点，profiling 正常统计，但是延迟指标依然是越来越来越高。Prometheus调用collect接口采集到的exec time时间始终是固定不变的。经debug排查到StoreCollector采集器内存hold的StoreMetricsHolder与CanalInstance实例中的引用已经不同啦，CanalIntance重启时已经被重建过一份新的实例，但是StoreMetricsHolder却没有保存到内存Hold中。原因是Map.putIfAbsent调用引起。

-【修复效果】
- 修复后，重启Server心跳正常推进，延迟瞬间降下来。梳理其他Collector代码都是调用Map.put，只有这里使用putIfAbsent可能是粗心导致吧。修复效果

* fix(canal/client-adapter): 修复adapter插件重试错误缺陷

-【缺陷描述】
  adapter运行期间，针对server或者Instance进行过一次重启或者断网测试，发现adapter通过client拉取数据报错以后持续打印错误日志，永远不得恢复。虽然adapter processor 针对错误有重试逻辑，但是client的tcp连接已经发生中断，且无法恢复。

-【问题定位】
  adapter processor 针对错误的处理比较粗，需要细分是从上游get message报错，还是下游sync data报错？
  如果是 get message报错，特别是tcp 断开，需要跳出重试，重新尝试重连。如果是下游 sync data 报错，再考虑重试写入。下游插件开发者自行保证连接重连机制、写入幂等。框架只提供重试。

- 说明：顺带更新一下 .gitignore 文件
 2025-05-16 15:12:42 +08:00
Sunxien®
e101287c25 fix(canal/startup.sh): 修复生产环境ZK启用sasl安全认证 (#5438) 
* fix(canal/startup.sh): 修复生产环境ZK启用sasl安全认证

- 如果生产环境zk启用sasl安全认证，canal-server/canal-adapter启动无法连接zk进行读写操作
- 默认支持用户将 jaas.conf 放置在 ./conf/ 目录下，启动脚本增加-Djava.security.auth.login.config 启动参数即可

* fix(canal/parent): upgrade h2 version

- runtime error: Value too long for column "CHARACTER VARYING" #5086

- Ref: https://github.com/alibaba/canal/issues/5086
 2025-05-16 15:04:02 +08:00
jianghang.loujh
c15129b0ac [maven-release-plugin] prepare for next development iteration 2025-01-16 16:07:08 +08:00
jianghang.loujh
0741ccda1e [maven-release-plugin] prepare release canal-1.1.8 2025-01-16 16:07:00 +08:00
Chongzhi Deng
44ed76ebb8 fix typo (#5299) 
Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
 2024-12-17 15:32:03 +08:00
JZ
fb3d2c6811 add compressionType,enableChunking for pulsar client , upgrade dependencies for security (#5191) 
* add support for listener name which is described in https://pulsar.apache.org/docs/next/concepts-multiple-advertised-listeners/

* upgrade for CVE-2023-20883  CVE-2022-1471  CVE-2023-20860

* add   enableChunking

* downgrade  pulsar client version

* fix

* 增加压缩算法
 2024-07-16 14:59:29 +08:00
jianghang.loujh
3439c48948 fixed pull #5175 , reformat 2024-07-16 14:57:53 +08:00
lwd-coding
9493ff733d fix: 同个instance配置多个group避免共享CanalMsgConsumer (#5175) 2024-06-26 15:49:48 +08:00
Tass
31530dc9af fix(adapter): add args into adapter/restart.sh (#5149) 
Signed-off-by: Tass <tasszz2k@gmail.com>
 2024-06-26 15:34:30 +08:00
ziyou
bbea802914 feat(es8): es8 支持证书认证 (#5106) 2024-04-28 09:54:53 +08:00
jianghang.loujh
348d218120 fixed issue #5029 , add deserializer message 2024-02-04 13:28:27 +08:00
xander-os
9baeff55e4 enhance clickhouse support (#4957) 2023-12-04 09:34:43 +08:00
jianghang.loujh
b322201889 [maven-release-plugin] prepare for next development iteration 2023-10-09 14:27:37 +08:00
jianghang.loujh
ea8949298f [maven-release-plugin] prepare release canal-1.1.7 2023-10-09 14:27:33 +08:00
jianghang.loujh
6440b744f2 fixed arm64 stack size too small, upgrade all to 1m 2023-10-08 18:55:06 +08:00
ymz
63dfbf65df es8x adapter (#4640) 2023-09-01 19:56:03 +08:00
jianghang.loujh
d90066e01b upgrade spring cloud & spring boot version 2023-09-01 19:51:49 +08:00
jianghang.loujh
7241af84f7 fixed issue #4742 , format 2023-08-22 17:52:38 +08:00
zard
159457cccd very very major：Adapter rdb bug fix (#4742) 
* 异常点描述1

* 问题描述

* bug fix

rdb yml文件 编码格式问题，导致所有outerAdapter初始化失败；
kafka消息空跑，数据全都丢失 问题 fix
 2023-08-22 17:50:12 +08:00
rewerma
cd798b71e9 add proxy for SPI (#4520) 
* 测试类报错

* add proxy for SPI
 2022-12-06 17:35:30 +08:00
jianghang.loujh
ef7e21e4b2 support jdk11 2022-09-01 18:27:19 +08:00
jianghang.loujh
7b8ff04d20 [maven-release-plugin] prepare for next development iteration 2022-05-24 10:26:12 +08:00
jianghang.loujh
7ee246ca5a [maven-release-plugin] prepare release canal-1.1.6 2022-05-24 10:26:06 +08:00
jianghang.loujh
99aefaa118 support mysql 8.0 transaction_payload_event 2022-05-23 23:23:52 +08:00
jianghang.loujh
eef1ee85fb fixed issue #4014, support java8 & java11 G1 GC 2022-05-23 18:23:31 +08:00
jianghang.loujh
05a52f1bd3 upgrade log4j version 2022-05-20 16:58:20 +08:00
jianghang.loujh
fb0f274ba8 fixed logger 2022-05-20 16:09:53 +08:00
jianghang.loujh
ee51f338e0 fixed outAdapterkey autoGen 2022-05-13 15:13:14 +08:00
zhuchao941
4315f203a2 repair issue #3401 (#3923) 
* 解决OutAdapter单例导致的问题，如 1.多个es7 outAdapter的多线程问题（共享同一个es bulk request），2.多个es7 outAdapter共用一份es配置问题

* 1. 优化adapter和监听器之间的代码结构
2. 监听器只处理匹配的配置文件
3. bugfix-es adapter配置文件回调时漏写esVersion
理论上一个adapter实例对应一个config配置文件

Co-authored-by: agapple <jianghang.loujh@alibaba-inc.com>
 2022-05-13 15:02:06 +08:00
jianghang.loujh
b12945d4a9 format yml 2022-05-13 14:39:57 +08:00
Treeyh
885f05fb6a 支持定义rocketmq client日志路径，及修复一些bug (#3229) 
* 修复接入阿里云RocketMq no route info for this topic 问题

* 支持deployer同步rocketmq时定义rocketmq client日志路径

* 支持deployer同步rocketmq时定义rocketmq client日志路径

* 回退guava版本号，解决adapter的java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()异常

* 解决adapter启动报com.alibaba.druid.pool.DruidDataSource cannot be cast to com.alibaba.druid.pool.DruidDataSource异常。

* 支持rocketmq client 配置日志路径

Co-authored-by: hai.yu <hai.yu@snowballtech.com>
Co-authored-by: agapple <jianghang.loujh@alibaba-inc.com>
 2022-05-13 14:39:29 +08:00
He Wang
11a5f2a0fa add druid stat filter (#3985) 2022-05-13 14:24:37 +08:00
zhangshaoning
bd3224eea1 Upgrade reference log4j package version (#4017) 2022-05-13 12:22:45 +08:00
ZhangJian He
571e7f4e25 fix ojdbc6 wrong corrinator (#4132) 2022-05-13 11:33:42 +08:00
jianghang.loujh
c42ba95eed fixed pulsarmq package 2021-10-09 18:06:34 +08:00
364102729
0a2254eeaf add tablestore adapter (#3754) 
* add tablestore adapter

* add tablestore adapter

* fix bug add add log when etl fail

* fix bug，tinyint(1)以及rowupdatechange

Co-authored-by: agapple <jianghang115@gmail.com>
 2021-10-09 16:06:13 +08:00
mikoto
4ce401d219 fix: 多destination的场景下只能生效一个 (#3772) 2021-10-09 15:38:26 +08:00
spark
63407dc570 canal adapter phoenix (#3290) 
* canal adapter phoenix

* canal adapter phoenix

* canal adapter phoenix

* canal adapter phoenix

* canal adapter phoenix

* canal adapter phoenix

* canal adapter phoenix

Co-authored-by: lihuav2017 <lihua@able-elec.com>
 2021-10-09 14:37:11 +08:00
agapple
5cc85ff2de [maven-release-plugin] prepare for next development iteration 2021-04-19 15:48:20 +08:00
agapple
64862503ba [maven-release-plugin] prepare release canal-1.1.5 2021-04-19 15:48:09 +08:00
许道杰
fede3b3d1d fix #3126 将hbase-client依赖修改为hbase-shaded-client (#3127) 2021-04-16 21:24:44 +08:00
hupeng
66408782b0 process()方法中，在retry循环中处理成功应该break (#3080) 
process()方法中，在retry循环中处理成功应该break
 2021-04-16 21:14:43 +08:00
许道杰
846c8870ae fix: issue #3076 (#3077) 2021-04-16 21:12:12 +08:00
agapple
b79af46054 merge code and fixed issue #2755 #2756 #2735 #2714 #2711 #2710 #2616 #2602 #2585 #2565 #2935 #2434 #2623 #2418 #2387 2020-08-22 13:07:24 +08:00