java.lang.Object | |
↳ | android.media.MediaCodec |
MediaCodec class can be used to access low-level media codec, i.e. encoder/decoder components.
MediaCodec is generally used like this:
MediaCodec codec = MediaCodec.createDecoderByType(type); codec.configure(format, ...); codec.start(); ByteBuffer[] inputBuffers = codec.getInputBuffers(); ByteBuffer[] outputBuffers = codec.getOutputBuffers(); for (;;) { int inputBufferIndex = codec.dequeueInputBuffer(timeoutUs); if (inputBufferIndex >= 0) { // fill inputBuffers[inputBufferIndex] with valid data ... codec.queueInputBuffer(inputBufferIndex, ...); } int outputBufferIndex = codec.dequeueOutputBuffer(timeoutUs); if (outputBufferIndex >= 0) { // outputBuffer is ready to be processed or rendered. ... codec.releaseOutputBuffer(outputBufferIndex, ...); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) { outputBuffers = codec.getOutputBuffers(); } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { // Subsequent data will conform to new format. MediaFormat format = codec.getOutputFormat(); ... } } codec.stop(); codec.release(); codec = null;Each codec maintains a number of input and output buffers that are referred to by index in API calls. The contents of these buffers is represented by the ByteBuffer[] arrays accessible through getInputBuffers() and getOutputBuffers(). After a successful call to
start()
the client "owns" neither
input nor output buffers, subsequent calls to
dequeueInputBuffer(long)
and
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
then transfer ownership from the codec
to the client.
The client is not required to resubmit/release buffers immediately to the codec, the sample code above simply does this for simplicity's sake.
Once the client has an input buffer available it can fill it with data
and submit it it to the codec via a call to
queueInputBuffer(int, int, int, long, int)
.
The codec in turn will return an output buffer to the client in response
to
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
. After the output buffer has been processed
a call to
releaseOutputBuffer(int, boolean)
will return it to the codec.
If a video surface has been provided in the call to
configure(MediaFormat, Surface, MediaCrypto, int)
,
releaseOutputBuffer(int, boolean)
optionally allows rendering of the buffer
to the surface.
Input buffers (for decoders) and Output buffers (for encoders) contain encoded data according to the format's type. For video types this data is all the encoded data representing a single moment in time, for audio data this is slightly relaxed in that a buffer may contain multiple encoded frames of audio. In either case, buffers do not start and end on arbitrary byte boundaries, this is not a stream of bytes, it's a stream of access units.
Most formats also require the actual data to be prefixed by a number
of buffers containing setup data, or codec specific data, i.e. the
first few buffers submitted to the codec object after starting it must
be codec specific data marked as such using the flag
BUFFER_FLAG_CODEC_CONFIG
in a call to
queueInputBuffer(int, int, int, long, int)
.
Codec specific data included in the format passed to
configure(MediaFormat, Surface, MediaCrypto, int)
(in ByteBuffer entries with keys "csd-0", "csd-1", ...) is automatically
submitted to the codec, this data MUST NOT be submitted explicitly by the
client.
Once the client reaches the end of the input data it signals the end of
the input stream by specifying a flag of
BUFFER_FLAG_END_OF_STREAM
in the call to
queueInputBuffer(int, int, int, long, int)
. The codec will continue to return output buffers
until it eventually signals the end of the output stream by specifying
the same flag (
BUFFER_FLAG_END_OF_STREAM
) on the BufferInfo returned in
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
.
In order to start decoding data that's not adjacent to previously submitted
data (i.e. after a seek) it is necessary to
flush()
the decoder.
Any input or output buffers the client may own at the point of the flush are
immediately revoked, i.e. after a call to
flush()
the client does not
own any buffers anymore.
Note that the format of the data submitted after a flush must not change,
flush does not support format discontinuities,
for this a full stop(), configure(), start() cycle is necessary.
Nested Classes | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
MediaCodec.BufferInfo | Per buffer metadata includes an offset and size specifying the range of valid data in the associated codec buffer. | |||||||||
|
MediaCodec.CryptoException | Thrown when a crypto error occurs while queueing a secure input buffer. | |||||||||
|
MediaCodec.CryptoInfo | Metadata describing the structure of a (at least partially) encrypted input sample. |
Constants | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
int | BUFFER_FLAG_CODEC_CONFIG | This indicated that the buffer marked as such contains codec initialization / codec specific data instead of media data. | |||||||||
int | BUFFER_FLAG_END_OF_STREAM | This signals the end of stream, i.e. | |||||||||
int | BUFFER_FLAG_SYNC_FRAME | This indicates that the buffer marked as such contains the data for a sync frame. | |||||||||
int | CONFIGURE_FLAG_ENCODE | If this codec is to be used as an encoder, pass this flag. | |||||||||
int | CRYPTO_MODE_AES_CTR | ||||||||||
int | CRYPTO_MODE_UNENCRYPTED | ||||||||||
int | INFO_OUTPUT_BUFFERS_CHANGED |
The output buffers have changed, the client must refer to the new
set of output buffers returned by
getOutputBuffers()
from
this point on.
|
|||||||||
int | INFO_OUTPUT_FORMAT_CHANGED | The output format has changed, subsequent data will follow the new format. | |||||||||
int | INFO_TRY_AGAIN_LATER |
If a non-negative timeout had been specified in the call
to
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
, indicates that the call timed out.
|
|||||||||
String | PARAMETER_KEY_REQUEST_SYNC_FRAME | Request that the encoder produce a sync frame "soon". | |||||||||
String | PARAMETER_KEY_SUSPEND | Temporarily suspend/resume encoding of input data. | |||||||||
String | PARAMETER_KEY_VIDEO_BITRATE | Change a video encoder's target bitrate on the fly. | |||||||||
int | VIDEO_SCALING_MODE_SCALE_TO_FIT | The content is scaled to the surface dimensions | |||||||||
int | VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING | The content is scaled, maintaining its aspect ratio, the whole surface area is used, content may be cropped |
Public Methods | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
Configures a component.
|
||||||||||
|
If you know the exact name of the component you want to instantiate
use this method to instantiate it.
|
||||||||||
|
Instantiate a decoder supporting input data of the given mime type.
|
||||||||||
|
Instantiate an encoder supporting output data of the given mime type.
|
||||||||||
|
Requests a Surface to use as the input to an encoder, in place of input buffers.
|
||||||||||
|
Returns the index of an input buffer to be filled with valid data
or -1 if no such buffer is currently available.
|
||||||||||
|
Dequeue an output buffer, block at most "timeoutUs" microseconds.
|
||||||||||
|
Flush both input and output ports of the component, all indices
previously returned in calls to
dequeueInputBuffer(long)
and
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
become invalid.
|
||||||||||
|
Get the codec info.
|
||||||||||
|
Call this after start() returns.
|
||||||||||
|
Get the component name.
|
||||||||||
|
Call this after start() returns and whenever dequeueOutputBuffer
signals an output buffer change by returning
INFO_OUTPUT_BUFFERS_CHANGED
|
||||||||||
|
Call this after dequeueOutputBuffer signals a format change by returning
INFO_OUTPUT_FORMAT_CHANGED
|
||||||||||
|
After filling a range of the input buffer at the specified index
submit it to the component.
|
||||||||||
|
Similar to
queueInputBuffer(int, int, int, long, int)
but submits a buffer that is
potentially encrypted.
|
||||||||||
|
Make sure you call this when you're done to free up any opened
component instance instead of relying on the garbage collector
to do this for you at some point in the future.
|
||||||||||
|
If you are done with a buffer, use this call to return the buffer to
the codec.
|
||||||||||
|
Communicate additional parameter changes to the component instance.
|
||||||||||
|
If a surface has been specified in a previous call to
configure(MediaFormat, Surface, MediaCrypto, int)
specifies the scaling mode to use.
|
||||||||||
|
Signals end-of-stream on input.
|
||||||||||
|
After successfully configuring the component, call start.
|
||||||||||
|
Finish the decode/encode session, note that the codec instance
remains active and ready to be
start()
ed again.
|
Protected Methods | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
|
Invoked when the garbage collector has detected that this instance is no longer reachable.
|
[Expand]
Inherited Methods
|
|||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
From class
java.lang.Object
|
This indicated that the buffer marked as such contains codec initialization / codec specific data instead of media data.
This signals the end of stream, i.e. no buffers will be available
after this, unless of course,
flush()
follows.
This indicates that the buffer marked as such contains the data for a sync frame.
If this codec is to be used as an encoder, pass this flag.
The output buffers have changed, the client must refer to the new
set of output buffers returned by
getOutputBuffers()
from
this point on.
The output format has changed, subsequent data will follow the new
format.
getOutputFormat()
returns the new format.
If a non-negative timeout had been specified in the call
to
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
, indicates that the call timed out.
Request that the encoder produce a sync frame "soon". Provide an Integer with the value 0.
Temporarily suspend/resume encoding of input data. While suspended input data is effectively discarded instead of being fed into the encoder. This parameter really only makes sense to use with an encoder in "surface-input" mode, as the client code has no control over the input-side of the encoder in that case. The value is an Integer object containing the value 1 to suspend or the value 0 to resume.
Change a video encoder's target bitrate on the fly. The value is an Integer object containing the new bitrate in bps.
The content is scaled to the surface dimensions
The content is scaled, maintaining its aspect ratio, the whole surface area is used, content may be cropped
Configures a component.
format | The format of the input data (decoder) or the desired format of the output data (encoder). |
---|---|
surface | Specify a surface on which to render the output of this decoder. |
crypto | Specify a crypto object to facilitate secure decryption of the media data. |
flags |
Specify
CONFIGURE_FLAG_ENCODE
to configure the
component as an encoder.
|
If you know the exact name of the component you want to instantiate
use this method to instantiate it. Use with caution.
Likely to be used with information obtained from
MediaCodecList
name | The name of the codec to be instantiated. |
---|
Instantiate a decoder supporting input data of the given mime type. The following is a partial list of defined mime types and their semantics:
type | The mime type of the input data. |
---|
Instantiate an encoder supporting output data of the given mime type.
type | The desired mime type of the output data. |
---|
Requests a Surface to use as the input to an encoder, in place of input buffers. This
may only be called after
configure(MediaFormat, Surface, MediaCrypto, int)
and before
start()
.
The application is responsible for calling release() on the Surface when done.
Returns the index of an input buffer to be filled with valid data or -1 if no such buffer is currently available. This method will return immediately if timeoutUs == 0, wait indefinitely for the availability of an input buffer if timeoutUs < 0 or wait up to "timeoutUs" microseconds if timeoutUs > 0.
timeoutUs | The timeout in microseconds, a negative timeout indicates "infinite". |
---|
Dequeue an output buffer, block at most "timeoutUs" microseconds. Returns the index of an output buffer that has been successfully decoded or one of the INFO_* constants below.
info | Will be filled with buffer meta data. |
---|---|
timeoutUs | The timeout in microseconds, a negative timeout indicates "infinite". |
Flush both input and output ports of the component, all indices
previously returned in calls to
dequeueInputBuffer(long)
and
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
become invalid.
Get the codec info. If the codec was created by createDecoderByType or createEncoderByType, what component is chosen is not known beforehand, and thus the caller does not have the MediaCodecInfo.
Get the component name. If the codec was created by createDecoderByType or createEncoderByType, what component is chosen is not known beforehand.
Call this after start() returns and whenever dequeueOutputBuffer
signals an output buffer change by returning
INFO_OUTPUT_BUFFERS_CHANGED
Call this after dequeueOutputBuffer signals a format change by returning
INFO_OUTPUT_FORMAT_CHANGED
After filling a range of the input buffer at the specified index
submit it to the component.
Many decoders require the actual compressed data stream to be
preceded by "codec specific data", i.e. setup data used to initialize
the codec such as PPS/SPS in the case of AVC video or code tables
in the case of vorbis audio.
The class
MediaExtractor
provides codec
specific data as part of
the returned track format in entries named "csd-0", "csd-1" ...
These buffers should be submitted using the flag
BUFFER_FLAG_CODEC_CONFIG
.
To indicate that this is the final piece of input data (or rather that
no more input data follows unless the decoder is subsequently flushed)
specify the flag
BUFFER_FLAG_END_OF_STREAM
.
index |
The index of a client-owned input buffer previously returned
in a call to
dequeueInputBuffer(long)
.
|
---|---|
offset | The byte offset into the input buffer at which the data starts. |
size | The number of bytes of valid input data. |
presentationTimeUs | The time at which this buffer should be rendered. |
flags |
A bitmask of flags
BUFFER_FLAG_SYNC_FRAME
,
BUFFER_FLAG_CODEC_CONFIG
or
BUFFER_FLAG_END_OF_STREAM
.
|
MediaCodec.CryptoException |
if a crypto object has been specified in
configure(MediaFormat, Surface, MediaCrypto, int)
|
---|
Similar to
queueInputBuffer(int, int, int, long, int)
but submits a buffer that is
potentially encrypted.
index |
The index of a client-owned input buffer previously returned
in a call to
dequeueInputBuffer(long)
.
|
---|---|
offset | The byte offset into the input buffer at which the data starts. |
info | Metadata required to facilitate decryption, the object can be reused immediately after this call returns. |
presentationTimeUs | The time at which this buffer should be rendered. |
flags |
A bitmask of flags
BUFFER_FLAG_SYNC_FRAME
,
BUFFER_FLAG_CODEC_CONFIG
or
BUFFER_FLAG_END_OF_STREAM
.
|
MediaCodec.CryptoException | if an error occurs while attempting to decrypt the buffer. An error code associated with the exception helps identify the reason for the failure. |
---|
Make sure you call this when you're done to free up any opened component instance instead of relying on the garbage collector to do this for you at some point in the future.
If you are done with a buffer, use this call to return the buffer to the codec. If you previously specified a surface when configuring this video decoder you can optionally render the buffer.
index |
The index of a client-owned output buffer previously returned
in a call to
dequeueOutputBuffer(MediaCodec.BufferInfo, long)
.
|
---|---|
render | If a valid surface was specified when configuring the codec, passing true renders this output buffer to the surface. |
Communicate additional parameter changes to the component instance.
If a surface has been specified in a previous call to
configure(MediaFormat, Surface, MediaCrypto, int)
specifies the scaling mode to use. The default is "scale to fit".
Signals end-of-stream on input. Equivalent to submitting an empty buffer with
BUFFER_FLAG_END_OF_STREAM
set. This may only be used with
encoders receiving input from a Surface created by
createInputSurface()
.
After successfully configuring the component, call start. On return you can query the component for its input/output buffers.
Invoked when the garbage collector has detected that this instance is no longer reachable. The default implementation does nothing, but this method can be overridden to free resources.
Note that objects that override
finalize
are significantly more expensive than
objects that don't. Finalizers may be run a long time after the object is no longer
reachable, depending on memory pressure, so it's a bad idea to rely on them for cleanup.
Note also that finalizers are run on a single VM-wide finalizer thread,
so doing blocking work in a finalizer is a bad idea. A finalizer is usually only necessary
for a class that has a native peer and needs to call a native method to destroy that peer.
Even then, it's better to provide an explicit
close
method (and implement
Closeable
), and insist that callers manually dispose of instances. This
works well for something like files, but less well for something like a
BigInteger
where typical calling code would have to deal with lots of temporaries. Unfortunately,
code that creates lots of temporaries is the worst kind of code from the point of view of
the single finalizer thread.
If you
must
use finalizers, consider at least providing your own
ReferenceQueue
and having your own thread process that queue.
Unlike constructors, finalizers are not automatically chained. You are responsible for
calling
super.finalize()
yourself.
Uncaught exceptions thrown by finalizers are ignored and do not terminate the finalizer thread. See Effective Java Item 7, "Avoid finalizers" for more.