51工具盒子

依楼听风雨
笑看云卷云舒,淡观潮起潮落

磐维数据库迁移系列-binlog 缺失导致无法增量迁移

磐维数据库,简称"PanWeiDB"。是中国移动信息技术中心首个基于中国本土开源数据库打造的面向ICT基础设施的自研数据库产品。其产品内核能力基于openGauss开源软件,并进一步提升了系统稳定性及多种数据库兼容性。

  • 环境信息

  • 问题描述

  • 解决过程

环境信息

  • 源库:MySQL 5.7

  • 目标库:磐维 2.0.2

  • dtp:2.0.2

问题描述

  • 使用 dtp 进行增量迁移,数据没同步过去;

  • incremental.log 报错:

    [ERROR][2024-06-13 11:36:44][cn.com.panwei.incremental.util.u.a(MysqlUtil.java:93)]查询mysql binlog初始状态异常:Access denied; you need (at least one of) the SUPER, REPLICATION CLIENT privilege(s) for this operation [INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gr(MysqlBinlogReader.java:1555)]collector send finish record to queue for reconnect! [WARN ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gr(MysqlBinlogReader.java:1561)]mysql采集端开始重连, task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1, binlogfile:mysql-bin.000003, position:136882081 [WARN ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gn(MysqlBinlogReader.java:441)]startRunListen 发生异常:sleep interrupted [INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.close(MysqlBinlogReader.java:190)]采集任务#taskID:【mysql_panwe12_ums_gogs#incremental#collect#incremental#1】, 【[采集端]子线程1[MySqlBinlog数据采集器]】关闭成功 [INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.start(MysqlBinlogReader.java:139)]采集任务#taskID:【mysql_panwe12_ums_gogs#incremental#collect#incremental#1】, 数据采集器线程【blc-10.27.47.9:3306】启动成功, 开始从MySqlBinlog采集数据 [INFO ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.d.gr(MysqlBinlogReader.java:1570)]mysql采集端重连成功,task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1 [WARN ][2024-06-13 20:02:05][cn.com.panwei.incremental.d.b.f.onDisconnect(MysqlBinlogReader.java:1631)]task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1 Stopped reading binlog, curBinlogFile:mysql-bin.000003, curBinlogPosition:136882081, currBeginPosition:136882081, lastAddRecordPosition:null [INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.d.gm(MysqlBinlogReader.java:412)]起始点: mysql-bin.000003:136882081 [INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.d.gm(MysqlBinlogReader.java:417)]GTID: 740e9574-125d-11ee-bec3-6c92bf3bb513:1-1097101,edda3576-f616-11ed-a9b2-005056bc3dd9:1-2812960 [INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.d.gm(MysqlBinlogReader.java:422)]采集任务#taskID:【mysql_panwe12_ums_gogs#incremental#collect#incremental#1】, 正在尝试建立binlog读取器连接【30000】后超时失败 [INFO ][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.f.onConnect(MysqlBinlogReader.java:1598)]task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1 Connected to MySQL binlog at 10.27.47.9:3306, starting at mysql-bin.000003:136882081 [ERROR][2024-06-13 20:02:10][cn.com.panwei.incremental.d.b.f.onCommunicationFailure(MysqlBinlogReader.java:1603)]task:mysql_panwe12_ums_gogs#incremental#collect#incremental#1 A communication failure event arrived com.github.shyiko.mysql.binlog.network.ServerException: Could not open log file at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:937) at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:606) at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:850) at java.lang.Thread.run(Thread.java:748)

  • 关键信息为倒数第五行的:

    com.github.shyiko.mysql.binlog.network.ServerException: Could not open log file

解决过程

  • 开始怀疑是权限问题,重新执行增量前置步骤后,问题仍然存在;

  • 实际原因是,源库有一个自动清理 binlog 的脚本,停止该定时任务后,不再报这个错误。


赞(5)
未经允许不得转载:工具盒子 » 磐维数据库迁移系列-binlog 缺失导致无法增量迁移