Android's Hidden (and Future) Camera APIs

With the release of the KitKat AOSP source code Google also exposed its plans for a new API for the camera -  package android.hardware.camera2.  The new interfaces and classes are marked as @hide and sit quietly in /frameworks/base/core/java/android/hardware/camera2. The @hide attribute excludes these classes from the automatic documentation generation and from the SDK.  The code is hidden because the API is not final and committed to by Google, but it is most likely to be quite similar in semantics, if not syntactically equivalent.  Anyone who's been watching the camera HAL changes in the last few Android releases will tell you that this new API is expected to become official anytime now - most likely in the L Android release.

The new API is inspired by the FCAM project and aims to give the application developer precise and flexible control over all aspects of the camera.  The old point & shoot paradigm which limits the camera to three dominating uses cases (preview, video, stills) is replaced by an API that abstracts the camera as a black box that produces streams of image frames in different formats and resolutions.  The camera "black box" is configured and controlled via an abstract canonical model of the camera processing pipeline controls and properties.  To understand the philosophy, motivation and details of this API, I think it is important to review the camera HAL (v3) documentation and to read the FCAM papers.  

If you're an Android camera application developer, then you would be wise to study the new API.  

Figure 1: the android.hardware.camera2 package

Besides studying the android.hardware.camera2 package, I looked for test and sample code in the AOSP to see how the API is used.

• ./frameworks/base/tests/Camera2Tests/SmartCamera/SimpleCamera/src/androidx/media/filterfw/samples/simplecamera/Camera2Source.java
• ./cts/tests/tests/hardware/src/android/hardware/camera2/cts/
• CameraCharacteristicsTest.java
• CameraDeviceTest.java
• CameraManagerTest.java
• CameraCaptureResultTest.java
• ImageReaderTest.java

The diagrams below depict a single simple use-case of setting up camera preview and issuing a few JPEG stills captures requests.  They are mostly self explanatory so I've included only a little text to describe them. If this is insufficient you can write your questions in the comments section below.

The process of discovering the cameras attached to the mobile device is depicted above.  Note the AvailabilityListener which monitors the dynamic connection and disconnection of cameras to the device.  I think it also monitors the use of camera objects by other applications.  Both features do not exist in the current (soon to be "legacy") API.

Figure 3: Preparing the surfaces

Before doing anything with the camera, you need to configure the output surfaces - that's where the camera will render the image frames.  Note the use of an ImageReader to obtain a Surface object to buffer JPEG formatted images.

Figure 4: Preview request

Preview is created by generating a repeating-request with a SurfaceView as the frame consumer.

Figure 5: Still capture request

Finally, when the user presses on the shutter button, the application issues a single capture request for a JPEG surface.  Buffers are held by the ImageReader until the application acquires them individually.

Summary

That was a very brief introduction to the android.hardware.camera2 package which I think will be officially included in the L Android release. I'm sure the legacy API will continue existing for a long time in order to support current camera applications.  However, you should consider learning the new API for the greater (and finer) camera control it provides.