Hadoop3 ShutdownHookManager visit closed ClassLoader

趕路人兒 2022-06-23 13:28:01 阅读数:877

hadoop3hadoopshutdownhookmanagervisitclosed

在yarn集群上啟動一個flink任務,拋出如下异常:

Exception in thread "Thread-5" java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.ensureInner(FlinkUserCodeClassLoaders.java:164)
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.getResource(FlinkUserCodeClassLoaders.java:183)
at org.apache.hadoop.conf.Configuration.getResource(Configuration.java:2737)
at org.apache.hadoop.conf.Configuration.getStreamReader(Configuration.java:2993)
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2952)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2925)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2805)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:1199)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1787)
at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
at org.apache.hadoop.util.ShutdownHookManager.shutdownExecutor(ShutdownHookManager.java:145)
at org.apache.hadoop.util.ShutdownHookManager.access$300(ShutdownHookManager.java:65)
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:102)

任務雖然可以正常運行,但是為了安全,還是google了一下這個錯誤,發現官方issue:

https://issues.apache.org/jira/browse/FLINK-19916
 

This is because Hadoop 3 starts asynchronous threads to execute some shutdown hooks.
These hooks are run after the job is executed, as a result, the classloader has been released, but in hooks, configuration still holds the released classloader, so it will fail to throw an exception in this asynchronous thread.

Now it doesn't affect our function, it just prints the exception stack on the console.

大致意思是:由於hadoop3引入了异步的線程來執行shutdown hook,該hook會在任務執行時運行,由於classloader已經被釋放,但是hook中仍然持有該classloader而跑出异常。該异常不影響正常功能,僅在控制臺打印日志。

網上提出一種解决方法:在 flink 配置文件裏 flink-conf.yaml設置 classloader.check-leaked-classloader: false

版权声明:本文为[趕路人兒]所创,转载请带上原文链接,感谢。 https://gsmany.com/2022/174/202206231252488216.html