51工具盒子在公司测试环境由于内存不足导致kafka服务自动停止，再次启动时java服务日志存在报错：
2024-09-18 10:31:35.727 WARN 23061 --- [consumer-0-C-1] o.a.k.c.NetworkClient : [Consumer clientId=consumer-ori-datasource-group-132, groupId=ori-datasource-group] 3 partitions have leader brokers without a matching listener, including [LO__8693UNN5W5U1V-0, LO_8693UNN5W5U1V-2, LO_8693UNN5W5U1V-1]
且无法消费。
在重新部署kafka后，java服务报错:
2024-09-19 11:21:16.362 INFO 12450 --- [main] o.a.k.c.u.AppInfoParser : Kafka version: 2.6.1
2024-09-19 11:21:16.362 INFO 12450 --- [main] o.a.k.c.u.AppInfoParser : Kafka commitId: 6b2021cd52659cef
2024-09-19 11:21:16.362 INFO 12450 --- [main] o.a.k.c.u.AppInfoParser : Kafka startTimeMs: 1726716076361
2024-09-19 11:21:16.377 INFO 12450 --- [main] o.a.k.c.Metadata : [Consumer clientId=consumer-morphling-group-1, groupId=morphling-group] Cluster ID: iz6BuPwmSYq-5AVvZ6qg9A
2024-09-19 11:22:16.378 WARN 12450 --- [main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'morphlingSceneCreateEventListener': Invocation of init method failed; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
2024-09-19 11:22:16.384 INFO 12450 --- [main] o.s.s.q.SchedulerFactoryBean : Shutting down Quartz Scheduler
2024-09-19 11:22:16.384 INFO 12450 --- [main] o.q.c.QuartzScheduler : Scheduler MINI-RUMTIME-QUARTZ-SCHEDULER_$_loli-platform-test21726716072751 shutting down.
2024-09-19 11:22:16.384 INFO 12450 --- [main] o.q.c.QuartzScheduler : Scheduler MINI-RUMTIME-QUARTZ-SCHEDULER_$_loli-platform-test21726716072751 paused.
2024-09-19 11:22:16.385 INFO 12450 --- [main] o.q.c.QuartzScheduler : Scheduler MINI-RUMTIME-QUARTZ-SCHEDULER_$_loli-platform-test21726716072751 shutdown complete.
2024-09-19 11:22:16.393 INFO 12450 --- [main] o.a.z.ZooKeeper : Session: 0x20b50402bf10e98 closed
2024-09-19 11:22:16.393 INFO 12450 --- [main-EventThread] o.a.z.ClientCnxn : EventThread shut down for session: 0x20b50402bf10e98
2024-09-19 11:22:16.398 INFO 12450 --- [main-EventThread] o.a.z.ClientCnxn : EventThread shut down for session: 0x10a719434c80e99
2024-09-19 11:22:16.398 INFO 12450 --- [main] o.a.z.ZooKeeper : Session: 0x10a719434c80e99 closed
2024-09-19 11:22:16.451 WARN 12450 --- [main] c._.i.l.f.LynxFlinkConfiguration : Lynx is disabled, method call to YarnClient should not be made:
2024-09-19 11:22:16.451 WARN 12450 --- [main] c._.i.l.f.LynxFlinkConfiguration : public void org.apache.hadoop.service.AbstractService.stop()([])
2024-09-19 11:22:16.477 INFO 12450 --- [Curator-Framework-0] o.a.c.f.i.CuratorFrameworkImpl : backgroundOperationsLoop exiting
2024-09-19 11:22:16.486 INFO 12450 --- [main-EventThread] o.a.z.ClientCnxn : EventThread shut down for session: 0x20b50402bf10e96
2024-09-19 11:22:16.486 INFO 12450 --- [main] o.a.z.ZooKeeper : Session: 0x20b50402bf10e96 closed
2024-09-19 11:22:16.488 INFO 12450 --- [main] o.a.k.c.p.KafkaProducer : [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
2024-09-19 11:22:16.497 INFO 12450 --- [main] c._.a.e.c.RestHighLevelClientFactory : Shutting down elasticsearch client
2024-09-19 11:22:16.524 INFO 12450 --- [main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2024-09-19 11:22:16.527 INFO 12450 --- [main] c.z.h.HikariDataSource : HikariPool-1 - Shutdown initiated...
2024-09-19 11:22:16.549 INFO 12450 --- [main] c.z.h.HikariDataSource : HikariPool-1 - Shutdown completed.
2024-09-19 11:22:16.608 INFO 12450 --- [main] c.b.e.w.c.DefaultService : Stopping service [com.bes.Service]
2024-09-19 11:22:16.665 INFO 12450 --- [main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2024-09-19 11:22:16.676 ERROR 12450 --- \[main\] o.s.b.SpringApplication : Application run failed
`org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'morphlingSceneCreateEventListener': Invocation of init method failed; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:139) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:414) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1763) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:593) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:847) ~[spring-beans-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877) ~[spring-context-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549) ~[spring-context-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:744) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:391) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1204) [spring-boot-2.1.12.RELEASE.jar!/:2.1.12.RELEASE]
at com._4paradigm.antifraud.gateway.GatewayApplication.main(GatewayApplication.java:28) [classes!/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_202]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_202]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_202]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_202]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52) [anti-fraud-gateway-release-3.5.3-4aa5db5-boot.jar:?]
Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata`
由于是测试环境，决定清理kafka及zookeeper数据。
清理数据
操作前建议停止调用kafka的服务，以免删除不掉。
检查topic
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181
删除topic
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181|awk '{print "./bin/kafka-topics.sh --delete --zookeeper 18.1.1.1:2181,18.2.2.2:2181 --topic",$1}'
删除zookeeper数据
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181|awk '{print "rmr /brokers/topics/"$1}'
./bin/kafka-topics.sh --list --zookeeper 18.1.1.1:2181,18.2.2.2:2181|awk '{print "rmr /admin/delete_topics/"$1}'
清理kafka数据
mv kafka-logs{,.bak}
清理数据及移除目录后，即可重启kafka，重新执行检查topic步骤，为空则表示完成。
