双亲委派模型与 Flink 的类加载策略
parent-first 类加载策略
public abstract class FlinkUserCodeClassLoader extends URLClassLoader {
public static final Consumer<Throwable> NOOP_EXCEPTION_HANDLER = classLoadingException -> {};
private final Consumer<Throwable> classLoadingExceptionHandler;
protected FlinkUserCodeClassLoader(URL[] urls, ClassLoader parent) {
this(urls, parent, NOOP_EXCEPTION_HANDLER);
}
protected FlinkUserCodeClassLoader(
URL[] urls,
ClassLoader parent,
Consumer<Throwable> classLoadingExceptionHandler) {
super(urls, parent);
this.classLoadingExceptionHandler = classLoadingExceptionHandler;
}
protected final Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
try {
return loadClassWithoutExceptionHandling(name, resolve);
} catch (Throwable classLoadingException) {
classLoadingExceptionHandler.accept(classLoadingException);
throw classLoadingException;
}
}
protected Class<?> loadClassWithoutExceptionHandling(String name, boolean resolve) throws ClassNotFoundException {
return super.loadClass(name, resolve);
}
}
static class ParentFirstClassLoader extends FlinkUserCodeClassLoader { ParentFirstClassLoader(URL[] urls, ClassLoader parent, Consumer<Throwable> classLoadingExceptionHandler) {
super(urls, parent, classLoadingExceptionHandler);
}
}
如果一个类加载器要加载一个类,它首先不会自己尝试加载这个类,而是把加载的请求委托给父加载器完成,所有的类加载请求最终都应该传递给最顶层的启动类加载器。只有当父加载器无法加载到这个类时,子加载器才会尝试自己加载。
child-first 类加载策略
public final class ChildFirstClassLoader extends FlinkUserCodeClassLoader {
private final String[] alwaysParentFirstPatterns;
public ChildFirstClassLoader(
URL[] urls,
ClassLoader parent,
String[] alwaysParentFirstPatterns,
Consumer<Throwable> classLoadingExceptionHandler) {
super(urls, parent, classLoadingExceptionHandler);
this.alwaysParentFirstPatterns = alwaysParentFirstPatterns;
}
@Override
protected synchronized Class<?> loadClassWithoutExceptionHandling(
String name,
boolean resolve) throws ClassNotFoundException {
// First, check if the class has already been loaded
Class<?> c = findLoadedClass(name);
if (c == null) {
// check whether the class should go parent-first
for (String alwaysParentFirstPattern : alwaysParentFirstPatterns) {
if (name.startsWith(alwaysParentFirstPattern)) {
return super.loadClassWithoutExceptionHandling(name, resolve);
}
}
try {
// check the URLs
c = findClass(name);
} catch (ClassNotFoundException e) {
// let URLClassLoader do it, which will eventually call the parent
c = super.loadClassWithoutExceptionHandling(name, resolve);
}
}
if (resolve) {
resolveClass(c);
}
return c;
}
@Override
public URL getResource(String name) {
// first, try and find it via the URLClassloader
URL urlClassLoaderResource = findResource(name);
if (urlClassLoaderResource != null) {
return urlClassLoaderResource;
}
// delegate to super
return super.getResource(name);
}
@Override
public Enumeration<URL> getResources(String name) throws IOException {
// first get resources from URLClassloader
Enumeration<URL> urlClassLoaderResources = findResources(name);
final List<URL> result = new ArrayList<>();
while (urlClassLoaderResources.hasMoreElements()) {
result.add(urlClassLoaderResources.nextElement());
}
// get parent urls
Enumeration<URL> parentResources = getParent().getResources(name);
while (parentResources.hasMoreElements()) {
result.add(parentResources.nextElement());
}
return new Enumeration<URL>() {
Iterator<URL> iter = result.iterator();
public boolean hasMoreElements() {
return iter.hasNext();
}
public URL nextElement() {
return iter.next();
}
};
}
}
-
调用 findLoadedClass() 方法检查全限定名 name 对应的类是否已经加载过,若没有加载过,再继续往下执行。 -
检查要加载的类是否以 alwaysParentFirstPatterns 集合中的前缀开头。如果是,则调用父类的对应方法,以 parent-first 的方式来加载它。 -
如果类不符合 alwaysParentFirstPatterns 集合的条件,就调用 findClass() 方法在用户代码中查找并获取该类的定义(该方法在 URLClassLoader 中有默认实现)。如果找不到,再 fallback 到父加载器来加载。 -
最后,若 resolve 参数为 true,就调用 resolveClass() 方法链接该类,最后返回对应的 Class 对象。
java.;
scala.;
org.apache.flink.;
com.esotericsoftware.kryo;
org.apache.hadoop.;
javax.annotation.;
org.slf4j;
org.apache.log4j;
org.apache.logging;
org.apache.commons.logging;
ch.qos.logback;
org.xml;
javax.xml;
org.apache.xerces;
org.w3c
-
classloader.parent-first-patterns.additional:除了上一个参数指定的类之外,用户如果有其他类以 child-first 模式会发生冲突,而希望以双亲委派模型来加载的话,可以额外指定(分号分隔)。
以上是关于 flink-conf.yaml 中提供的控制类加载策略的参数 classloader.resolve-order 含义的理解和分享,希望对大家有所启发和帮助~
原文链接:
https://www.jianshu.com/p/bc7309b03407