Android 中几个重要的View
SurfaceView从Android 1.0(API level 1)时就有 。它继承自类View,因此它本质上是一个View。但与普通View不同的是,它有自己的Surface。我们知道,一般的Activity包含的多个View会组成View hierachy的树形结构,只有最顶层的DecorView,也就是根结点视图,才是对WMS可见的。这个DecorView在WMS中有一个对应的WindowState。相应地,在SF中对应的Layer。而SurfaceView自带一个Surface,这个Surface在WMS中有自己对应的WindowState,在SF中也会有自己的Layer。如下图所示:
GLSurfaceView从Android 1.5(API level 3)开始加入,作为SurfaceView的补充。它可以看作是SurfaceView的一种典型使用模式。在SurfaceView的基础上,它加入了EGL的管理,并自带了渲染线程。另外它定义了用户需要实现的Render接口,提供了用Strategy pattern更改具体Render行为的灵活性。作为GLSurfaceView的Client,只需要将实现了渲染函数的Renderer的实现类设置给GLSurfaceView即可。如:
public class TriangleActivity extends Activity { protected void onCreate(Bundle savedInstanceState) { mGLView = new GLSurfaceView(this); mGLView.setRenderer(new RendererImpl(this));
首先,VideoDumpView是GLSurfaceView的继承类。在构造函数VideoDumpView()中会创建VideoDumpRenderer,也就是GLSurfaceView.Renderer的实例,然后调setRenderer()将之设成GLSurfaceView的Renderer。
109 public VideoDumpView(Context context) { ... 116 mRenderer = new VideoDumpRenderer(context); 117 setRenderer(mRenderer); 118 }
随后,GLSurfaceView中的GLThread启动,创建EGL环境后回调VideoDumpRenderer中的onSurfaceCreated()。
519 public void onSurfaceCreated(GL10 glUnused, EGLConfig config) { ... 551 // Create our texture. This has to be done each time the surface is created. 552 int[] textures = new int[1]; 553 GLES20.glGenTextures(1, textures, 0); 554 555 mTextureID = textures[0]; 556 GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mTextureID); ... 575 mSurface = new SurfaceTexture(mTextureID); 576 mSurface.setOnFrameAvailableListener(this); 577 578 Surface surface = new Surface(mSurface); 579 mMediaPlayer.setSurface(surface);
SurfaceTexture的参数为GLES接口函数glGenTexture()得到的纹理对象id。在初始化函数SurfaceTexture_init()中,先创建GLConsumer和相应的BufferQueue,再将它们的指针通过JNI放到SurfaceTexture的Java层对象成员中。
230 static void SurfaceTexture_init(JNIEnv* env, jobject thiz, jboolean isDetached, 231 jint texName, jboolean singleBufferMode, jobject weakThiz) 232{ ... 235 BufferQueue::createBufferQueue(&producer, &consumer); ... 242 sp<GLConsumer> surfaceTexture; 243 if (isDetached) { 244 surfaceTexture = new GLConsumer(consumer, GL_TEXTURE_EXTERNAL_OES, 245 true, true); 246 } else { 247 surfaceTexture = new GLConsumer(consumer, texName, 248 GL_TEXTURE_EXTERNAL_OES, true, true); 249 } ... 256 SurfaceTexture_setSurfaceTexture(env, thiz, surfaceTexture); 257 SurfaceTexture_setProducer(env, thiz, producer); ... 266 sp<JNISurfaceTextureContext> ctx(new JNISurfaceTextureContext(env, weakThiz, 267 clazz)); 268 surfaceTexture->setFrameAvailableListener(ctx); 269 SurfaceTexture_setFrameAvailableListener(env, thiz, ctx);
由于直接的Listener在Java层,而触发者在Native层,因此需要从Native层回调到Java层。这里通过JNISurfaceTextureContext当了跳板。JNISurfaceTextureContext的onFrameAvailable()起到了Native和Java的桥接作用:
180 void JNISurfaceTextureContext::onFrameAvailable() ... 184 env->CallStaticVoidMethod(mClazz, fields.postEvent, mWeakThiz);
回到onSurfaceCreated(),接下来创建供外部生产者使用的Surface类。Surface的构造函数之一带有参数SurfaceTexture。
133 public Surface(SurfaceTexture surfaceTexture) { ... 140 setNativeObjectLocked(nativeCreateFromSurfaceTexture(surfaceTexture));
它实际上是把SurfaceTexture中创建的BufferQueue的Producer接口实现类拿出来后创建了相应的Surface类。
135 static jlong nativeCreateFromSurfaceTexture(JNIEnv* env, jclass clazz, 136 jobject surfaceTextureObj) { 137 sp<IGraphicBufferProducer> producer(SurfaceTexture_getProducer(env, surfaceTextureObj)); ... 144 sp<Surface> surface(new Surface(producer, true));
与此同时,GLSurfaceView中的GLThread也在运行,它会调用到VideoDumpRenderer的绘制函数onDrawFrame()。
372 public void onDrawFrame(GL10 glUnused) { ... 377 if (updateSurface) { ... 380 mSurface.updateTexImage(); 381 mSurface.getTransformMatrix(mSTMatrix); 382 updateSurface = false; ... 394 // Activate the texture. 395 GLES20.glActiveTexture(GLES20.GL_TEXTURE0); 396 GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, mTextureID); ... 421 // Draw a rectangle and render the video frame as a texture on it. 422 GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4); ... 429 DumpToFile(frameNumber);
上面讲了SurfaceTexture,下面看看TextureView是如何工作的。还是从例子着手,Android的关于TextureView的官方文档(http://developer.android.com/reference/android/view/TextureView.html)给了一个简洁的例子LiveCameraActivity。它它可以将Camera中的内容放在View中进行显示。在onCreate()函数中首先创建TextureView,再将Activity(实现了TextureView.SurfaceTextureListener接口)传给TextureView,用于监听SurfaceTexture准备好的信号。
protected void onCreate(Bundle savedInstanceState) { ... mTextureView = new TextureView(this); mTextureView.setSurfaceTextureListener(this); ... }
TextureView的构造函数并不做主要的初始化工作。主要的初始化工作是在getHardwareLayer()中,而这个函数是在其基类View的draw()中调用。TextureView重载了这个函数:
348 HardwareLayer getHardwareLayer() { ... 358 mLayer = mAttachInfo.mHardwareRenderer.createTextureLayer(); 359 if (!mUpdateSurface) { 360 // Create a new SurfaceTexture for the layer. 361 mSurface = new SurfaceTexture(false); 362 mLayer.setSurfaceTexture(mSurface); 363 } 364 mSurface.setDefaultBufferSize(getWidth(), getHeight()); 365 nCreateNativeWindow(mSurface); 366 367 mSurface.setOnFrameAvailableListener(mUpdateListener, mAttachInfo.mHandler); 368 369 if (mListener != null && !mUpdateSurface) { 370 mListener.onSurfaceTextureAvailable(mSurface, getWidth(), getHeight()); 371 } ... 390 applyUpdate(); 391 applyTransformMatrix(); 392 393 return mLayer; 394 }
前面提到初始化中会调用onSurfaceTextureAvailable()这个回调函数。在它的实现中,TextureView的使用者就可以将准备好的SurfaceTexture传给数据源模块,供数据源输出之用。如:
public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) { mCamera = Camera.open(); ... mCamera.setPreviewTexture(surface); mCamera.startPreview(); ... }
看一下setPreviewTexture()的实现,其中把SurfaceTexture中初始化时创建的GraphicBufferProducer拿出来传给Camera模块。
576static void android_hardware_Camera_setPreviewTexture(JNIEnv *env, 577 jobject thiz, jobject jSurfaceTexture) ... 585 producer = SurfaceTexture_getProducer(env, jSurfaceTexture); ... 594 if (camera->setPreviewTarget(producer) != NO_ERROR) {
到这里,一切都初始化地差不多了。接下来当内容流有新图像可用,TextureView会被通知到(通过SurfaceTexture.OnFrameAvailableListener接口)。SurfaceTexture.OnFrameAvailableListener是SurfaceTexture有新内容来时的回调接口。TextureView中的mUpdateListener实现了该接口:
755 public void onFrameAvailable(SurfaceTexture surfaceTexture) { 756 updateLayer(); 757 invalidate(); 758 }
可以看到其中会调用updateLayer()函数,然后通过invalidate()函数申请更新UI。updateLayer()会设置mUpdateLayer标志位。这样,当下次VSync到来时,Choreographer通知App通过重绘View hierachy。在UI重绘函数performTranversals()中,作为View hierachy的一分子,TextureView的draw()函数被调用,其中便会相继调用applyUpdate()和HardwareLayer的updateSurfaceTexture()函数。
138 public void updateSurfaceTexture() { 139 nUpdateSurfaceTexture(mFinalizer.get()); 140 mRenderer.pushLayerUpdate(this); 141 }