chenjl/canal
1
0
Fork 0
mirror of https://github.com/alibaba/canal.git synced 2026-11-04 00:30:15 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,725 Commits 13 Branches 38 Tags
master
Commit Graph

403 Commits

Author SHA1 Message Date
jianghang.loujh
e20e442446 fix(client-adapter/phoenix): 修复 Phoenix ETL params 参数 SQL 注入漏洞 
将 SYS_TIME 默认条件和 etlCondition 中的用户输入改为 PreparedStatement
占位符绑定，避免直接拼接 SQL 导致注入。
 2026-07-15 14:59:08 +08:00
jianghang.loujh
cf97b2ae31 fix: mitigate SnakeYAML deserialization RCE and enforce default password change 
1. Use SafeConstructor in ApplicationConfigMonitor to block arbitrary
   object instantiation via YAML type tags (e.g. !!javax.script.ScriptEngineManager).
2. Detect default admin password (admin/123456) on login and enforce
   password change before allowing any other API operations (code 50015).

🤖 Generated with [Qoder][https://qoder.com]
 2026-04-29 18:07:02 +08:00
jianghang.loujh
1a265c8960 fixed issue #5474 , scaled_float -> BigDecimal 2026-02-12 17:33:39 +08:00
jianghang.loujh
24877fdf03 merge code and format 2026-02-12 17:32:33 +08:00
jiaowc
c0de979595 修复同步mysql数据到es7和es8 使用es别名报错 Not found the mapping info of index 问题 (#5560) 2026-02-12 15:21:39 +08:00
usedssl
fcadf564ba fix(canal/client-adapter): 修复adapter插件全量同步sqlservice错误(#5564) (#5566) 2026-02-12 15:13:37 +08:00
jianghang.loujh
4ded2869c9 merge code 2026-01-28 12:06:35 +08:00
sunxien
dcea8afc93 refactor(canal): 升级JVM参数 (#5544) 
- 重点升级JVM参数

- 重构SessionHandler代码, 处理错误请求会响应两次包；
 2026-01-28 11:20:19 +08:00
jianghang.loujh
d18bde71c6 merge and support polardb-x recycle_bin binlog format 2025-12-25 11:47:36 +08:00
sunxien
5a5e60d371 feat(canal/adapter): adapter支持prometheus暴露jvm指标 (#5549) 
- adapter 默认提供prometheus暴露JVM指标

- 指定 maven-antrun-plugin 未指定版本号，默认是 3.0.0，但是配置中使用 <tasks> 标签，编译报错。
   - 降级版本号，显式指定 1.8 版本
   - 或者把配置中的 <tasks> 标签改成 <target>
 2025-12-25 11:41:08 +08:00
jianghang.loujh
556d998db1 adjust preferIPv4Stack=false, support ipv6 2025-08-05 16:42:00 +08:00
jianghang.loujh
25501db3ac migrate mvn deploy from OSSRH to maven central 2025-08-05 14:51:14 +08:00
Sunxien®
50a570290a fix(canal/client-adapter): 修复adapter插件重试错误缺陷 (#5427) 
* fix(canal/prometheus): 修复延迟指标的缺陷

-【缺陷描述】
   Admin控制台修改canal.properties配置后，Server自动进行重启，在Grafan上观测到同步实例的PUT/GET/ACK延迟居高不下。

-【原因定位】
   源库心跳正常触发，MemoryStoreWithBuffer正常推进位点，profiling 正常统计，但是延迟指标依然是越来越来越高。Prometheus调用collect接口采集到的exec time时间始终是固定不变的。经debug排查到StoreCollector采集器内存hold的StoreMetricsHolder与CanalInstance实例中的引用已经不同啦，CanalIntance重启时已经被重建过一份新的实例，但是StoreMetricsHolder却没有保存到内存Hold中。原因是Map.putIfAbsent调用引起。

-【修复效果】
- 修复后，重启Server心跳正常推进，延迟瞬间降下来。梳理其他Collector代码都是调用Map.put，只有这里使用putIfAbsent可能是粗心导致吧。修复效果

* fix(canal/client-adapter): 修复adapter插件重试错误缺陷

-【缺陷描述】
  adapter运行期间，针对server或者Instance进行过一次重启或者断网测试，发现adapter通过client拉取数据报错以后持续打印错误日志，永远不得恢复。虽然adapter processor 针对错误有重试逻辑，但是client的tcp连接已经发生中断，且无法恢复。

-【问题定位】
  adapter processor 针对错误的处理比较粗，需要细分是从上游get message报错，还是下游sync data报错？
  如果是 get message报错，特别是tcp 断开，需要跳出重试，重新尝试重连。如果是下游 sync data 报错，再考虑重试写入。下游插件开发者自行保证连接重连机制、写入幂等。框架只提供重试。

- 说明：顺带更新一下 .gitignore 文件
 2025-05-16 15:12:42 +08:00
Sunxien®
e101287c25 fix(canal/startup.sh): 修复生产环境ZK启用sasl安全认证 (#5438) 
* fix(canal/startup.sh): 修复生产环境ZK启用sasl安全认证

- 如果生产环境zk启用sasl安全认证，canal-server/canal-adapter启动无法连接zk进行读写操作
- 默认支持用户将 jaas.conf 放置在 ./conf/ 目录下，启动脚本增加-Djava.security.auth.login.config 启动参数即可

* fix(canal/parent): upgrade h2 version

- runtime error: Value too long for column "CHARACTER VARYING" #5086

- Ref: https://github.com/alibaba/canal/issues/5086
 2025-05-16 15:04:02 +08:00
jianghang.loujh
e21e98aa83 fixed CanalAdmin instanceLog and polardbx columnar test 2025-03-19 15:51:27 +08:00
nakeiven
c998b3f100 Feature/adapter postgresql 20250124 (#5390) 
* adapter tcp模式mysql数据全量同步至postgresql时 报错
fix issue 2146

* fix issue 1980

* feat 增加 当dbType 为 postgresql时，返回双引号(避免表名为postgresql的关键字时，sql查询报错)

---------

Co-authored-by: guodong.zhang <zgd8988@126.com>
 2025-01-24 17:14:41 +08:00
jianghang.loujh
c15129b0ac [maven-release-plugin] prepare for next development iteration 2025-01-16 16:07:08 +08:00
jianghang.loujh
0741ccda1e [maven-release-plugin] prepare release canal-1.1.8 2025-01-16 16:07:00 +08:00
Chongzhi Deng
44ed76ebb8 fix typo (#5299) 
Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
 2024-12-17 15:32:03 +08:00
jianghang.loujh
fe836f7515 fixed issue #5336 #5332 , etl bugfix 2024-12-17 15:29:51 +08:00
JZ
fb3d2c6811 add compressionType,enableChunking for pulsar client , upgrade dependencies for security (#5191) 
* add support for listener name which is described in https://pulsar.apache.org/docs/next/concepts-multiple-advertised-listeners/

* upgrade for CVE-2023-20883  CVE-2022-1471  CVE-2023-20860

* add   enableChunking

* downgrade  pulsar client version

* fix

* 增加压缩算法
 2024-07-16 14:59:29 +08:00
jianghang.loujh
3439c48948 fixed pull #5175 , reformat 2024-07-16 14:57:53 +08:00
Yazid Lee
9373429015 fix: resource loading failure caused by incorrect parameter order in StringUtils.startsWithIgnoreCase (#5173) 2024-06-26 15:53:24 +08:00
lwd-coding
9493ff733d fix: 同个instance配置多个group避免共享CanalMsgConsumer (#5175) 2024-06-26 15:49:48 +08:00
Tass
31530dc9af fix(adapter): add args into adapter/restart.sh (#5149) 
Signed-off-by: Tass <tasszz2k@gmail.com>
 2024-06-26 15:34:30 +08:00
jianghang.loujh
7bb619cf12 fixed #5154,avoid spring RCE 2024-06-06 15:29:17 +08:00
jianghang.loujh
aa05d8e2a0 reformat 2024-05-22 09:46:13 +08:00
ziyou
bbea802914 feat(es8): es8 支持证书认证 (#5106) 2024-04-28 09:54:53 +08:00
jianghang.loujh
059f21208d fixed issue #5066 , es7.x npe 2024-03-18 14:19:45 +08:00
jianghang.loujh
8ca900377a merge & reformat code 2024-03-08 11:58:46 +08:00
jackzlz
6dffcc8573 yml解析支持占位符 (#5045) 2024-03-08 11:56:30 +08:00
wbxlll
711971599f fix issue client-adapter.rdb模块在转换时间时丢失精度 #5052 (#5053) 
* fix the issue that client-adapter.rdb module loses precision when converting microsencond time

* fix the issue that client-adapter.rdb module loses precision when converting to java.sql.Time or java.sql.Timestamp

* fix the issue that client-adapter.rdb module loses precision when converting to java.sql.Time or java.sql.Timestamp
 2024-03-08 11:50:53 +08:00
jianghang.loujh
b9ed5caf05 format 2024-02-04 13:35:10 +08:00
jianghang.loujh
348d218120 fixed issue #5029 , add deserializer message 2024-02-04 13:28:27 +08:00
zoemak
e2a64daad8 fixed issue #5037 ,使用client-adapter.rdb模块进行ETL同步，源库为MySQL，数据库名有短横线，读取源数据时报SQL异常 (#5044) 2024-02-04 13:07:51 +08:00
jianghang.loujh
93dac2d335 format clickhouse adapter code 2023-12-04 09:46:23 +08:00
xander-os
9baeff55e4 enhance clickhouse support (#4957) 2023-12-04 09:34:43 +08:00
jianghang.loujh
b322201889 [maven-release-plugin] prepare for next development iteration 2023-10-09 14:27:37 +08:00
jianghang.loujh
ea8949298f [maven-release-plugin] prepare release canal-1.1.7 2023-10-09 14:27:33 +08:00
dack-su
ba924b9f1d update org.postgresql:postgresql 42.3.3 to 42.5.1 (#4898) 2023-10-09 13:46:20 +08:00
jianghang.loujh
6440b744f2 fixed arm64 stack size too small, upgrade all to 1m 2023-10-08 18:55:06 +08:00
cooper lee
cda330e49e Fix Configuration property name '-id' is not valid on canal 1.1.7 alpha 3 #4879 (#4887) 
* Fix Configuration property name '-id' is not valid

* Fix Configuration property name '-id' is not valid
读取原配置文件
 2023-09-21 10:53:47 +08:00
weirhp
d623bea71a 解决rdb下有多个配制，会导致配置相互覆盖的问题 (#4560) 
恩baseProperties这个变量是共享的，不能修改这个变量的属性值。要不后续的配置都会改变。关联issue为4355
 2023-09-04 17:23:26 +08:00
jianghang.loujh
2786e06ccf format es8 2023-09-01 19:58:23 +08:00
ymz
63dfbf65df es8x adapter (#4640) 2023-09-01 19:56:03 +08:00
jianghang.loujh
d90066e01b upgrade spring cloud & spring boot version 2023-09-01 19:51:49 +08:00
jianghang.loujh
681d03fbe7 upgrade dependency version for CVE 2023-09-01 13:59:05 +08:00
jianghang.loujh
7241af84f7 fixed issue #4742 , format 2023-08-22 17:52:38 +08:00
zard
159457cccd very very major：Adapter rdb bug fix (#4742) 
* 异常点描述1

* 问题描述

* bug fix

rdb yml文件 编码格式问题，导致所有outerAdapter初始化失败；
kafka消息空跑，数据全都丢失 问题 fix
 2023-08-22 17:50:12 +08:00
jianghang.loujh
d8a70d0c2f fixed issue #4822 , fix connection close 2023-08-22 17:32:42 +08:00