| Top | |
|
|
|
| #define | GST_VIDEO_DECODER_FLOW_NEED_DATA |
| #define | GST_VIDEO_DECODER_MAX_ERRORS |
| #define | GST_VIDEO_DECODER_SINK_NAME |
| #define | GST_VIDEO_DECODER_SRC_NAME |
| struct | GstVideoDecoder |
| struct | GstVideoDecoderClass |
This base class is for video decoders turning encoded data into raw video frames.
The GstVideoDecoder base class and derived subclasses should cooperate as follows:
Configuration
Initially, GstVideoDecoder calls start when the decoder element
is activated, which allows the subclass to perform any global setup.
GstVideoDecoder calls set_format to inform the subclass of caps
describing input video data that it is about to receive, including
possibly configuration data.
While unlikely, it might be called more than once, if changing input
parameters require reconfiguration.
Incoming data buffers are processed as needed, described in Data Processing below.
GstVideoDecoder calls stop at end of all processing.
Data processing
The base class gathers input data, and optionally allows subclass to parse this into subsequently manageable chunks, typically corresponding to and referred to as 'frames'.
Each input frame is provided in turn to the subclass' handle_frame
callback.
The ownership of the frame is given to the handle_frame callback.
If codec processing results in decoded data, the subclass should call
gst_video_decoder_finish_frame to have decoded data pushed.
downstream. Otherwise, the subclass must call
gst_video_decoder_drop_frame, to allow the base class to do timestamp
and offset tracking, and possibly to requeue the frame for a later
attempt in the case of reverse playback.
Shutdown phase
The GstVideoDecoder class calls stop to inform the subclass that data
parsing will be stopped.
Additional Notes
Seeking/Flushing
When the pipeline is seeked or otherwise flushed, the subclass is
informed via a call to its reset callback, with the hard parameter
set to true. This indicates the subclass should drop any internal data
queues and timestamps and prepare for a fresh set of buffers to arrive
for parsing and decoding.
End Of Stream
At end-of-stream, the subclass parse function may be called some final
times with the at_eos parameter set to true, indicating that the element
should not expect any more data to be arriving, and it should parse and
remaining frames and call gst_video_decoder_have_frame() if possible.
The subclass is responsible for providing pad template caps for
source and sink pads. The pads need to be named "sink" and "src". It also
needs to provide information about the ouptput caps, when they are known.
This may be when the base class calls the subclass' set_format
function,
though it might be during decoding, before calling
gst_video_decoder_finish_frame
. This is done via
gst_video_decoder_set_output_state
The subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, the base class provides limited framerate based interpolation.
Similarly, the base class provides some limited (legacy) seeking support
if specifically requested by the subclass, as full-fledged support
should rather be left to upstream demuxer, parser or alike. This simple
approach caters for seeking and duration reporting using estimated input
bitrates. To enable it, a subclass should call
gst_video_decoder_set_estimate_rate
to enable handling of incoming
byte-streams.
The base class provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, the subclass should then be prepared for the parsing and frame processing stage to occur separately (in normal forward processing, the latter immediately follows the former), The subclass also needs to ensure the parsing stage properly marks keyframes, unless it knows the upstream elements will do so properly for incoming data.
The bare minimum that a functional subclass needs to implement is:
Provide pad templates
Inform the base class of output caps via
gst_video_decoder_set_output_state
Parse input data, if it is not considered packetized from upstream
Data will be provided to parse which should invoke
gst_video_decoder_add_to_frame and gst_video_decoder_have_frame to
separate the data belonging to each video frame.
Accept data in handle_frame and provide decoded results to
gst_video_decoder_finish_frame, or call gst_video_decoder_drop_frame.
#define GST_VIDEO_DECODER_ERROR(el, w, domain, code, text, debug, ret)
Utility function that video decoder elements can use in case they encountered
a data processing error that may be fatal for the current "data unit" but
need not prevent subsequent decoding. Such errors are counted and if there
are too many, as configured in the context's max_errors, the pipeline will
post an error message and the application will be requested to stop further
media processing. Otherwise, it is considered a "glitch" and only a warning
is logged. In either case, ret
is set to the proper value to
return to upstream/caller (indicating either GST_FLOW_ERROR or GST_FLOW_OK).
el |
the base video decoder element that generates the error |
|
w |
element defined weight of the error, added to error count |
|
domain |
like CORE, LIBRARY, RESOURCE or STREAM (see gstreamer-GstGError) |
|
code |
error code defined for that domain (see gstreamer-GstGError) |
|
text |
the message to display (format string and args enclosed in parentheses) |
|
debug |
debugging information for the message (format string and args enclosed in parentheses) |
|
ret |
variable to receive return value |
#define GST_VIDEO_DECODER_SINK_PAD(obj) (((GstVideoDecoder *) (obj))->sinkpad)
Gives the pointer to the sink GstPad object of the element.
#define GST_VIDEO_DECODER_SRC_PAD(obj) (((GstVideoDecoder *) (obj))->srcpad)
Gives the pointer to the source GstPad object of the element.
#define GST_VIDEO_DECODER_STREAM_LOCK(decoder) g_rec_mutex_lock (&GST_VIDEO_DECODER (decoder)->stream_lock)
Obtain a lock to protect the decoder function from concurrent access.
#define GST_VIDEO_DECODER_STREAM_UNLOCK(decoder) g_rec_mutex_unlock (&GST_VIDEO_DECODER (decoder)->stream_lock)
Release the lock that protects the decoder function from concurrent access.
#define GST_VIDEO_DECODER_INPUT_SEGMENT(obj) (GST_VIDEO_DECODER_CAST (obj)->input_segment)
Gives the segment of the element.
#define GST_VIDEO_DECODER_OUTPUT_SEGMENT(obj) (GST_VIDEO_DECODER_CAST (obj)->output_segment)
Gives the segment of the element.
void gst_video_decoder_add_to_frame (GstVideoDecoder *decoder,int n_bytes);
Removes next n_bytes
of input data and adds it to currently parsed frame.
GstBuffer *
gst_video_decoder_allocate_output_buffer
(GstVideoDecoder *decoder);
Helper function that allocates a buffer to hold a video frame for decoder
's
current GstVideoCodecState.
You should use gst_video_decoder_allocate_output_frame() instead of this
function, if possible at all.
GstFlowReturn gst_video_decoder_allocate_output_frame (GstVideoDecoder *decoder,GstVideoCodecFrame *frame);
Helper function that allocates a buffer to hold a video frame for decoder
's
current GstVideoCodecState. Subclass should already have configured video
state and set src pad caps.
The buffer allocated here is owned by the frame and you should only keep references to the frame, not the buffer.
void gst_video_decoder_get_allocator (GstVideoDecoder *decoder,GstAllocator **allocator,GstAllocationParams *params);
Lets GstVideoDecoder sub-classes to know the memory allocator
used by the base class and its params
.
Unref the allocator
after use it.
decoder |
||
allocator |
the GstAllocator used. |
[out][allow-none][transfer full] |
params |
the
GstAllocatorParams of |
[out][allow-none][transfer full] |
GstBufferPool *
gst_video_decoder_get_buffer_pool (GstVideoDecoder *decoder);
the instance of the GstBufferPool used by the decoder; free it after use it.
[transfer full]
GstFlowReturn gst_video_decoder_drop_frame (GstVideoDecoder *dec,GstVideoCodecFrame *frame);
Similar to gst_video_decoder_finish_frame(), but drops frame
in any
case and posts a QoS message with the frame's details on the bus.
In any case, the frame is considered finished and released.
GstFlowReturn gst_video_decoder_finish_frame (GstVideoDecoder *decoder,GstVideoCodecFrame *frame);
frame
should have a valid decoded data buffer, whose metadata fields
are then appropriately set according to frame data and pushed downstream.
If no output data is provided, frame
is considered skipped.
In any case, the frame is considered finished and released.
After calling this function the output buffer of the frame is to be considered read-only. This function will also change the metadata of the buffer.
void gst_video_decoder_release_frame (GstVideoDecoder *dec,GstVideoCodecFrame *frame);
Similar to gst_video_decoder_drop_frame(), but simply releases frame
without any processing other than removing it from list of pending frames,
after which it is considered finished and released.
Since: 1.2.2
gboolean
gst_video_decoder_negotiate (GstVideoDecoder *decoder);
Negotiate with downstream elements to currently configured GstVideoCodecState. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.
GstVideoCodecFrame * gst_video_decoder_get_frame (GstVideoDecoder *decoder,int frame_number);
Get a pending unfinished GstVideoCodecFrame
GList *
gst_video_decoder_get_frames (GstVideoDecoder *decoder);
Get all pending unfinished GstVideoCodecFrame
GstClockTimeDiff gst_video_decoder_get_max_decode_time (GstVideoDecoder *decoder,GstVideoCodecFrame *frame);
Determines maximum possible decoding time for frame
that will
allow it to decode and arrive in time (as determined by QoS events).
In particular, a negative result means decoding in time is no longer possible
and should therefore occur as soon/skippy as possible.
GstVideoCodecFrame *
gst_video_decoder_get_oldest_frame (GstVideoDecoder *decoder);
Get the oldest pending unfinished GstVideoCodecFrame
gboolean
gst_video_decoder_get_packetized (GstVideoDecoder *decoder);
Queries whether input data is considered packetized or not by the base class.
gsize
gst_video_decoder_get_pending_frame_size
(GstVideoDecoder *decoder);
Returns the number of bytes previously added to the current frame
by calling gst_video_decoder_add_to_frame().
Since: 1.4
gdouble
gst_video_decoder_get_qos_proportion (GstVideoDecoder *decoder);
Since: 1.0.3
GstFlowReturn
gst_video_decoder_have_frame (GstVideoDecoder *decoder);
Gathers all data collected for currently parsed frame, gathers corresponding
metadata and passes it along for further processing, i.e. handle_frame
.
void gst_video_decoder_get_latency (GstVideoDecoder *decoder,GstClockTime *min_latency,GstClockTime *max_latency);
Query the configured decoder latency. Results will be returned via
min_latency
and max_latency
.
void gst_video_decoder_set_latency (GstVideoDecoder *decoder,GstClockTime min_latency,GstClockTime max_latency);
Lets GstVideoDecoder sub-classes tell the baseclass what the decoder latency is. Will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.
gint
gst_video_decoder_get_estimate_rate (GstVideoDecoder *dec);
GstVideoCodecState *
gst_video_decoder_get_output_state (GstVideoDecoder *decoder);
Get the GstVideoCodecState currently describing the output stream.
void gst_video_decoder_set_estimate_rate (GstVideoDecoder *dec,gboolean enabled);
Allows baseclass to perform byte to time estimated conversion.
GstVideoCodecState * gst_video_decoder_set_output_state (GstVideoDecoder *decoder,GstVideoFormat fmt,guint width,guint height,GstVideoCodecState *reference);
Creates a new GstVideoCodecState with the specified fmt
, width
and height
as the output state for the decoder.
Any previously set output state on decoder
will be replaced by the newly
created one.
If the subclass wishes to copy over existing fields (like pixel aspec ratio,
or framerate) from an existing GstVideoCodecState, it can be provided as a
reference
.
If the subclass wishes to override some fields from the output state (like pixel-aspect-ratio or framerate) it can do so on the returned GstVideoCodecState.
The new output state will only take effect (set on pads and buffers) starting
from the next call to gst_video_decoder_finish_frame().
decoder |
||
fmt |
||
width |
The width in pixels |
|
height |
The height in pixels |
|
reference |
An optional reference GstVideoCodecState. |
[allow-none][transfer none] |
void gst_video_decoder_set_max_errors (GstVideoDecoder *dec,gint num);
Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. You can set -1 for never returning fatal errors. Default is set to GST_VIDEO_DECODER_MAX_ERRORS.
The '-1' option was added in 1.4
void gst_video_decoder_set_packetized (GstVideoDecoder *decoder,gboolean packetized);
Allows baseclass to consider input data as packetized or not. If the
input is packetized, then the parse
method will not be called.
gboolean
gst_video_decoder_get_needs_format (GstVideoDecoder *dec);
Queries decoder required format handling.
Since: 1.4
void gst_video_decoder_set_needs_format (GstVideoDecoder *dec,gboolean enabled);
Configures decoder format needs. If enabled, subclass needs to be negotiated with format caps before it can process any data. It will then never be handed any data before it has been configured. Otherwise, it might be handed data without having been configured and is then expected being able to do so either by default or based on the input data.
Since: 1.4
void gst_video_decoder_merge_tags (GstVideoDecoder *decoder,const GstTagList *tags,GstTagMergeMode mode);
Sets the audio decoder tags and how they should be merged with any
upstream stream tags. This will override any tags previously-set
with gst_audio_decoder_merge_tags().
Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.
MT safe.
decoder |
||
tags |
a GstTagList to merge, or NULL to unset previously-set tags. |
[allow-none] |
mode |
the GstTagMergeMode to use, usually GST_TAG_MERGE_REPLACE |
GstCaps * gst_video_decoder_proxy_getcaps (GstVideoDecoder *decoder,GstCaps *caps,GstCaps *filter);
Returns caps that express caps
(or sink template caps if caps
== NULL)
restricted to resolution/format/... combinations supported by downstream
elements.
Since: 1.6
#define GST_VIDEO_DECODER_FLOW_NEED_DATA GST_FLOW_CUSTOM_SUCCESS
Returned while parsing to indicate more data is needed.
#define GST_VIDEO_DECODER_MAX_ERRORS 10
Default maximum number of errors tolerated before signaling error.
#define GST_VIDEO_DECODER_SINK_NAME "sink"
The name of the templates for the sink pad.
#define GST_VIDEO_DECODER_SRC_NAME "src"
The name of the templates for the source pad.
struct GstVideoDecoderClass {
gboolean (*open) (GstVideoDecoder *decoder);
gboolean (*close) (GstVideoDecoder *decoder);
gboolean (*start) (GstVideoDecoder *decoder);
gboolean (*stop) (GstVideoDecoder *decoder);
GstFlowReturn (*parse) (GstVideoDecoder *decoder,
GstVideoCodecFrame *frame,
GstAdapter *adapter,
gboolean at_eos);
gboolean (*set_format) (GstVideoDecoder *decoder,
GstVideoCodecState * state);
gboolean (*reset) (GstVideoDecoder *decoder,
gboolean hard);
GstFlowReturn (*finish) (GstVideoDecoder *decoder);
GstFlowReturn (*handle_frame) (GstVideoDecoder *decoder,
GstVideoCodecFrame *frame);
gboolean (*sink_event) (GstVideoDecoder *decoder,
GstEvent *event);
gboolean (*src_event) (GstVideoDecoder *decoder,
GstEvent *event);
gboolean (*negotiate) (GstVideoDecoder *decoder);
gboolean (*decide_allocation) (GstVideoDecoder *decoder, GstQuery *query);
gboolean (*propose_allocation) (GstVideoDecoder *decoder, GstQuery * query);
gboolean (*flush) (GstVideoDecoder *decoder);
gboolean (*sink_query) (GstVideoDecoder *decoder,
GstQuery *query);
gboolean (*src_query) (GstVideoDecoder *decoder,
GstQuery *query);
GstCaps* (*getcaps) (GstVideoDecoder *decoder,
GstCaps *filter);
GstFlowReturn (*drain) (GstVideoDecoder *decoder);
gboolean (*transform_meta) (GstVideoDecoder *decoder,
GstVideoCodecFrame *frame,
GstMeta * meta);
};
Subclasses can override any of the available virtual methods or not, as
needed. At minimum handle_frame
needs to be overridden, and set_format
and likely as well. If non-packetized input is supported or expected,
parse
needs to be overridden as well.
Optional. Called when the element changes to GST_STATE_READY. Allows opening external resources. |
||
Optional. Called when the element changes to GST_STATE_NULL. Allows closing external resources. |
||
Optional. Called when the element starts processing. Allows opening external resources. |
||
Optional. Called when the element stops processing. Allows closing external resources. |
||
Required for non-packetized input. Allows chopping incoming data into manageable units (frames) for subsequent decoding. |
||
Notifies subclass of incoming data format (caps). |
||
Optional. Allows subclass (decoder) to perform post-seek semantics reset. Deprecated. |
||
Optional. Called to request subclass to dispatch any pending remaining data at EOS. Sub-classes can refuse to decode new data after. |
||
Provides input data frame to subclass. |
||
Optional. Event handler on the sink pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler. |
||
Optional. Event handler on the source pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler. |
||
Optional. Negotiate with downstream and configure buffer pools, etc. Subclasses should chain up to the parent implementation to invoke the default handler. |
||
Optional. Setup the allocation parameters for allocating output buffers. The passed in query contains the result of the downstream allocation query. Subclasses should chain up to the parent implementation to invoke the default handler. |
||
Optional. Propose buffer allocation parameters for upstream elements. Subclasses should chain up to the parent implementation to invoke the default handler. |
||
Optional. Flush all remaining data from the decoder without pushing it downstream. Since: 1.2 |
||
Optional. Query handler on the sink pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since 1.4 |
||
Optional. Query handler on the source pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since 1.4 |
||
Optional. Allows for a custom sink getcaps implementation. If not implemented, default returns gst_video_decoder_proxy_getcaps applied to sink template caps. |
||
Optional. Called to request subclass to decode any data it can at this point, but that more data may arrive after. (e.g. at segment end). Sub-classes should be prepared to handle new data afterward, or seamless segment processing will break. Since: 1.6 |
||
Optional. Transform the metadata on the input buffer to the
output buffer. By default this method is copies all meta without
tags and meta with only the "video" tag. subclasses can
implement this method and return |