Documentation/Maemo 5 Developer Guide/Architecture/Multimedia Domain - Revision history

imported_>joerg_rw: /* Input Event Sounds */ rypo

2014-09-30T17:26:58Z

Input Event Sounds: rypo

← Older revision		Revision as of 17:26, 30 September 2014
Line 250:		Line 250:
	===Input Event Sounds===		===Input Event Sounds===

	The input event sounds module is using the Xserver xtest (http://www.xfree86.org/current/xtestlib.pdf) extension to produce input ~~even~~ feedback via libcanberra. The input-sound module is started with the XSession as a separate process.		The input event sounds module is using the Xserver xtest (http://www.xfree86.org/current/xtestlib.pdf) extension to produce input event feedback via libcanberra. The input-sound module is started with the XSession as a separate process.

	==Metadata Subsystem==		==Metadata Subsystem==

imported_>joerg_rw: /* FMTX Middleware */ -not, logic

2013-10-10T07:35:58Z

FMTX Middleware: -not, logic

← Older revision		Revision as of 07:35, 10 October 2013
Line 223:		Line 223:
	FMTX middleware provides a daemon for controlling the [[N900 FM radio transmitter\|FM Transmitter]]. The daemon listens to commands from clients via dbus system interface. The frequency of the transmitter is controlled via Video4Linux2 interface. Note that the transmitter must be unmuted before changing frequency. This is because the device is muted by default and when the device is muted, it's not powered. Other settings are controlled by sysfs files in directory <code>/sys/bus/i2c/devices/2-0063/</code>.		FMTX middleware provides a daemon for controlling the [[N900 FM radio transmitter\|FM Transmitter]]. The daemon listens to commands from clients via dbus system interface. The frequency of the transmitter is controlled via Video4Linux2 interface. Note that the transmitter must be unmuted before changing frequency. This is because the device is muted by default and when the device is muted, it's not powered. Other settings are controlled by sysfs files in directory <code>/sys/bus/i2c/devices/2-0063/</code>.

	The wire of the headset acts as an antenna, boosting fmtx transmission power over allowed limits. Therefore the daemon is monitoring plugged devices and powers the transmitter down, if the headset is ~~not~~ connected.		The wire of the headset acts as an antenna, boosting fmtx transmission power over allowed limits. Therefore the daemon is monitoring plugged devices and powers the transmitter down, if the headset is connected.
	GConf: system/fmtx/:		GConf: system/fmtx/:
	*Bool enabled		*Bool enabled

imported_>joerg_rw: /* FMTX Middleware */ licence

2013-10-10T07:34:31Z

FMTX Middleware: licence

← Older revision		Revision as of 07:34, 10 October 2013
Line 227:		Line 227:
	*Bool enabled		*Bool enabled
	*Int frequency		*Int frequency

			==== licence ====
			Nokia

	===Audio/ Video Synchronization===		===Audio/ Video Synchronization===

imported_>joerg_rw: /* EAP and AEP */ licence

2013-10-10T07:33:09Z

EAP and AEP: licence

← Older revision		Revision as of 07:33, 10 October 2013
Line 216:		Line 216:
	AEP (Audio Enhancements Package) is a full duplex speech audio enhancement package including echo cancellation, background noise suppression, DRC, AGC, etc.		AEP (Audio Enhancements Package) is a full duplex speech audio enhancement package including echo cancellation, background noise suppression, DRC, AGC, etc.
	Both EAP and AEP are implemented as a PulseAudio module.		Both EAP and AEP are implemented as a PulseAudio module.
			====licence====
			Nokia

	===FMTX Middleware===		===FMTX Middleware===

imported_>pali: /* libplayback */

2012-07-24T18:38:34Z

libplayback

← Older revision		Revision as of 18:38, 24 July 2012
Line 278:		Line 278:
	: [http://maemo.org/packages/view/libplayback-1-0/ libplayback]		: [http://maemo.org/packages/view/libplayback-1-0/ libplayback]
	;Documentation and example code		;Documentation and example code
			: [http://maemo.org/api_refs/5.0/5.0-final/libplayback-1/ Documentation]
	: [http://talk.maemo.org/showthread.php?t=67157 Thread on libplayback]		: [http://talk.maemo.org/showthread.php?t=67157 Thread on libplayback]

imported_>xkr47: /* PulseAudio */ add pulseaudio-utils package link

2011-03-04T19:14:48Z

PulseAudio: add pulseaudio-utils package link

← Older revision		Revision as of 19:14, 4 March 2011
Line 196:		Line 196:
	: GPL/LGPL		: GPL/LGPL
	;Packages		;Packages
	: [http://maemo.org/packages/view/pulseaudio/ pulseaudio]		: [http://maemo.org/packages/view/pulseaudio/ pulseaudio] [http://maemo.org/packages/view/pulseaudio-utils/ pulseaudio-utils]

	====Public interface provided by PulseAudio====		====Public interface provided by PulseAudio====

imported_>amigadave: reformat tables

2011-03-02T10:50:42Z

reformat tables

← Older revision		Revision as of 10:50, 2 March 2011
Line 43:		Line 43:

	{\| class="wikitable"		{\| class="wikitable"
			\|+ Public interfaces provided by GStreamer
			\|-
	! Interface name !! Description		! Interface name !! Description
	\|-		\|-
Line 120:		Line 122:

	{\| class="wikitable"		{\| class="wikitable"
			\|+ Public interface provided by camera daemon
			\|-
	! Interface name !! Description		! Interface name !! Description
	\|-		\|-
Line 168:		Line 172:

	{\| class="wikitable"		{\| class="wikitable"
			\|+ Publix interface provided by ALSA library
			\|-
	! Interface name !! Description		! Interface name !! Description
	\|-		\|-
Line 195:		Line 201:

	{\| class="wikitable"		{\| class="wikitable"
			\|+ Public interface provided by PulseAudio
			\|-
	! Interface name !! Description		! Interface name !! Description
	\|-		\|-
Line 271:		Line 279:
	;Documentation and example code		;Documentation and example code
	: [http://talk.maemo.org/showthread.php?t=67157 Thread on libplayback]		: [http://talk.maemo.org/showthread.php?t=67157 Thread on libplayback]
	~~'''~~Public interface provided by libplayback~~'''~~
			====Public interface provided by libplayback====

	{\| class="wikitable"		{\| class="wikitable"
			\|+ Public interface provided by libplayback
			\|-
	! Interface name !! Description		! Interface name !! Description
	\|-		\|-

imported_>thomasjfox: /* libplayback */

2011-02-04T22:14:21Z

libplayback

← Older revision		Revision as of 22:14, 4 February 2011
Line 262:		Line 262:

	;Purpose		;Purpose
	: It can be used by media application to synchronize their playback state.		: It can be used by media application to synchronize their playback state. This includes audio playback when the silent profile is active.
	;Responsibilities		;Responsibilities
	: Provides an interface for media playback management		: Provides an interface for media playback management
Line 269:		Line 269:
	;Packages		;Packages
	: [http://maemo.org/packages/view/libplayback-1-0/ libplayback]		: [http://maemo.org/packages/view/libplayback-1-0/ libplayback]
			;Documentation and example code
			: [http://talk.maemo.org/showthread.php?t=67157 Thread on libplayback]
	'''Public interface provided by libplayback'''		'''Public interface provided by libplayback'''
	{\| class="wikitable"		{\| class="wikitable"

imported_>amigadave: /* FMTX Middleware */ add link, use


2010-09-02T14:21:12Z
FMTX Middleware:  add link, use <code>

				
				
				
				
				
				← Older revision
				Revision as of 14:21, 2 September 2010
				
Line 211:
Line 211:
===FMTX Middleware=== ===FMTX Middleware===



FMTX middleware provides a daemon for controlling the FM Transmitter. The daemon listens to commands from clients via dbus system interface. The frequency of the transmitter is controlled via Video4Linux2 interface. Note that the transmitter must be unmuted before changing frequency. This is because the device is muted by default and when the device is muted, it's not powered. Other settings are controlled by sysfs files in directory /sys/bus/i2c/devices/2-0063/. FMTX middleware provides a daemon for controlling the [[N900 FM radio transmitter|FM Transmitter]]. The daemon listens to commands from clients via dbus system interface. The frequency of the transmitter is controlled via Video4Linux2 interface. Note that the transmitter must be unmuted before changing frequency. This is because the device is muted by default and when the device is muted, it's not powered. Other settings are controlled by sysfs files in directory <code>/sys/bus/i2c/devices/2-0063/</code>.



The wire of the headset acts as an antenna, boosting fmtx transmission power over allowed limits. Therefore the daemon is monitoring plugged devices and powers the transmitter down, if the headset is not connected. The wire of the headset acts as an antenna, boosting fmtx transmission power over allowed limits. Therefore the daemon is monitoring plugged devices and powers the transmitter down, if the headset is not connected.



imported_>amigadave: reformat images
2010-08-18T13:06:21Z
reformat images

				
				
				
				
				
				← Older revision
				Revision as of 13:06, 18 August 2010
				
Line 7:
Line 7:






[[Image:OMAP_architecture.png|800px]] [[Image:OMAP_architecture.png|800px|alt=Diagram of top-level multimedia architecture|OMAP multimedia architecture]]






  Major challenge for the Multimedia framework is to provide the same interfaces like on desktop Linux for different hardware platforms. In order to achieve this objective, the whole audio processing is moved to the ARM side (ALSA / PulseAudio) and OpenMAX IL is introduced to abstract the hardware specific components (codecs and other hardware accelerated elements).  
Major challenge for the Multimedia framework is to provide the same interfaces like on desktop Linux for different hardware platforms. In order to achieve this objective, the whole audio processing is moved to the ARM side (ALSA / PulseAudio) and OpenMAX IL is introduced to abstract 	the hardware specific components (codecs and other hardware accelerated elements).   



==Video subsystem== ==Video subsystem==



[[Image:mmf_video_decomposition.png]] [[Image:mmf_video_decomposition.png|frame|center|alt=UML diagram of video subsystem|Video subsystem]]






===Video Codecs=== ===Video Codecs===

Most of the video encoders and decoders are wrapped under the OpenMAX IL interface [OMAX], which abstracts the codec specific implementation (which in general can run on ARM or DSP). All the OpenMAX IL video codecs will be running on the DSP side in order to best exploit the HW acceleration provided by the OMAP 3430 platform. The DSP based OpenMAX IL components are loaded by TIs IL-Core, which in turn uses LCML and the DSP-Bridge. The ARM based OpenMAX IL are loaded via Bellagio IL-Core. Both IL-Cores are used by the gst-openmax bridge. The GStreamer framework resides on the ARM side.   Most of the video encoders and decoders are wrapped under the OpenMAX IL interface [OMAX], which abstracts the codec specific implementation (which in general can run on ARM or DSP). All the OpenMAX IL video codecs will be running on the DSP side in order to best exploit the HW acceleration provided by the OMAP 3430 platform. The DSP based OpenMAX IL components are loaded by TIs IL-Core, which in turn uses LCML and the DSP-Bridge. The ARM based OpenMAX IL are loaded via Bellagio IL-Core. Both IL-Cores are used by the gst-openmax bridge. The GStreamer framework resides on the ARM side.  



Video post processing is performed on DSS screen accelerator. DSS is used to do the colorspace conversion, the scaling and composition, including overlays. A separate external graphics accelerator is used to refresh the screen. In case of need (complicated use cases) the scaling and colorspace conversion can be done on the ARM side as well, but that is not recommened as it is not optimized. Video post processing is performed on DSS screen accelerator. DSS is used to do the colorspace conversion, the scaling and composition, including overlays. A separate external graphics accelerator is used to refresh the screen. In case of need (complicated use cases) the scaling and colorspace conversion can be done on the ARM side as well, but that is not recommened as it is not optimized. A/V synchronization is done on the ARM, using an audio clock that is based on information from the audio interface.
A/V synchronization is done on the ARM, using an audio clock that is based on information from the audio interface. 



The communication between ARM and DSP software is provided by the TI DSP bridge. Any messages or data buffers exchanged between the ARM and DSP go through it. This layer can be regarded as transparent from Multimedia Architecture point of view and hence it will not be described in this document. The communication between ARM and DSP software is provided by the TI DSP bridge. Any messages or data buffers exchanged between the ARM and DSP go through it. This layer can be regarded as transparent from Multimedia Architecture point of view and hence it will not be described in this document.



===GStreamer=== ===GStreamer===

GStreamer is a crossplatform media framework, covering most multimedia applications use cases from playback, to streaming and imaging. It is a huge collection of objects, interfaces, libraries and plugins. From the application point of view it is just one utility library that can be used by applications to process media streams. The library interface is actually a facade to a versatile collection of dynamic modules that implement the actual functionality. GStreamer core hides the complexity of timing issues, synchronization, buffering, threading, streaming and other functionalities that are needed to produce usable media application.   GStreamer is a crossplatform media framework, covering most multimedia applications use cases from playback, to streaming and imaging. It is a huge collection of objects, interfaces, libraries and plugins. From the application point of view it is just one utility library that can be used by applications to process media streams. The library interface is actually a facade to a versatile collection of dynamic modules that implement the actual functionality. GStreamer core hides the complexity of timing issues, synchronization, buffering, threading, streaming and other functionalities that are needed to produce usable media application.  



Line 39:
Line 39:
*GStreamer provides good modularity and flexibility.  Hence, building applications on GStreamer in short time is possible. *GStreamer provides good modularity and flexibility.  Hence, building applications on GStreamer in short time is possible.
*GStreamer is LGPL and it allows Multimedia Project to combine GStreamer with proprietary software. *GStreamer is LGPL and it allows Multimedia Project to combine GStreamer with proprietary software.




====Public interfaces provided by GStreamer==== ====Public interfaces provided by GStreamer====
Line 46:
Line 45:
! Interface name !! Description ! Interface name !! Description
|- |-
|GStreamer API||Interface for Multimedia applications, VOIP etc | GStreamer API || Interface for Multimedia applications, VOIP etc
|- |-
|playbin2||Recommended high level element for playback. | <code>playbin2</code> || Recommended high level element for playback.
|- |-
|uricodebin||Recommended high level element for decoding. | <code>uricodebin</code> || Recommended high level element for decoding.
|- |-
|tagreadbin||Recommended high level element for fast metadata reading. | <code>tagreadbin</code> || Recommended high level element for fast metadata reading.
|- |-
|camerabin||Recommended high level element for camera application. | <code>camerabin</code> || Recommended high level element for camera application.
|} |}



===OpenMAX IL=== ===OpenMAX IL===

OpenMAX is an effort to provide an industry standard for a multimedia API. The standard defines 3 layers – OpenMAX DL (Development Layer), OpenMAX IL (Integration Layer) and OpenMAX AL (Application Layer). DL is a vendor specific and optional component. IL is the layer that interfaces with IL components (e.g. codecs). We will integrate TI IL-Core for the DSP components and Bellagio IL-Core for ARM components. Neither of the cores uses DL. OpenMAX is an effort to provide an industry standard for a multimedia API. The standard defines 3 layers – OpenMAX DL (Development Layer), OpenMAX IL (Integration Layer) and OpenMAX AL (Application Layer). DL is a vendor specific and optional component. IL is the layer that interfaces with IL components (e.g. codecs). We will integrate TI IL-Core for the DSP components and Bellagio IL-Core for ARM components. Neither of the cores uses DL.
We use GStreamer’s gomx module to transparently make OpenMAX IL components available to any GStreamer application. We use GStreamer’s gomx module to transparently make OpenMAX IL components available to any GStreamer application.
Line 80:
Line 80:



==Imaging subsystem== ==Imaging subsystem==

The multimedia framework also provides support for imaging applications. The subsystem is illustrated below. The multimedia framework also provides support for imaging applications. The subsystem is illustrated below.



[[Image:Imaging_decomposition.png|800px]] [[Image:Imaging_decomposition.png|800px|alt=UML diagram of imaging subsystem|Imaging subsystem]]






===Camera Source=== ===Camera Source===
The GstV4L2CamSrc is a fork of the GstV4l2Src plugin. The reason for the fork is that the original plugin has lots of extra code for handing various V4L2 devices (such as tuner cards) and that made the code quite complex.  
The <code>GstV4L2CamSrc</code> is a fork of the <code>GstV4l2Src</code> plugin. The reason for the fork is that the original plugin has lots of extra code for handing various V4L2 devices (such as tuner cards) and that made the code quite complex.



====GStreamer V4L2 Camera Source==== ====GStreamer V4L2 Camera Source====
Line 120:
Line 122:
! Interface name !! Description ! Interface name !! Description
|- |-
|V4L2 ioctl||Extensions for V4L2 protocol | V4L2 ioctl || Extensions for V4L2 protocol
|} |}



==Audio Subsystem== ==Audio Subsystem==



[[Image:Audio_decomposition.png]] [[Image:Audio_decomposition.png|frame|center|alt=UML diagram of audio subsystem|audio subsystem]]






Line 224:
Line 226:
MMF delivers parts of the notification mechanism. The subsytem is illustrated below: MMF delivers parts of the notification mechanism. The subsytem is illustrated below:



[[Image:Notification_decomposition.png]] [[Image:Notification_decomposition.png|frame|center|alt=UML diagram of notification subsystem|Notification subsystem]]






Line 241:
Line 243:
The multimedia framework supports the search engine when indexing media files. The subsystem is illustrated below: The multimedia framework supports the search engine when indexing media files. The subsystem is illustrated below:



[[Image:Metadata_decomposition.png]] [[Image:Metadata_decomposition.png|frame|center|alt=UML diagram of metadata subsystem|Metadata subsystem]]






Line 252:
Line 254:
It is neither user friendly nor always possible to run several multimedia use cases at the same time. To provide a predictable and stable behavior, multimedia components interact with a system wide policy component to keep concurrent multimedia use cases under control. The subsystem is illustrated below. The policy engine is based on the OHM framework. It is dynamically configurable with scripting and a prolog based rule database. It is neither user friendly nor always possible to run several multimedia use cases at the same time. To provide a predictable and stable behavior, multimedia components interact with a system wide policy component to keep concurrent multimedia use cases under control. The subsystem is illustrated below. The policy engine is based on the OHM framework. It is dynamically configurable with scripting and a prolog based rule database.



[[Image:Policy_decomposition.png]] [[Image:Policy_decomposition.png|frame|center|alt=UML diagram of policy subsystem|Policy subsystem]]

← Older revision		Revision as of 14:21, 2 September 2010
Line 211:		Line 211:
	===FMTX Middleware===		===FMTX Middleware===

	FMTX middleware provides a daemon for controlling the FM Transmitter. The daemon listens to commands from clients via dbus system interface. The frequency of the transmitter is controlled via Video4Linux2 interface. Note that the transmitter must be unmuted before changing frequency. This is because the device is muted by default and when the device is muted, it's not powered. Other settings are controlled by sysfs files in directory /sys/bus/i2c/devices/2-0063/.		FMTX middleware provides a daemon for controlling the [[N900 FM radio transmitter\|FM Transmitter]]. The daemon listens to commands from clients via dbus system interface. The frequency of the transmitter is controlled via Video4Linux2 interface. Note that the transmitter must be unmuted before changing frequency. This is because the device is muted by default and when the device is muted, it's not powered. Other settings are controlled by sysfs files in directory <code>/sys/bus/i2c/devices/2-0063/</code>.

	The wire of the headset acts as an antenna, boosting fmtx transmission power over allowed limits. Therefore the daemon is monitoring plugged devices and powers the transmitter down, if the headset is not connected.		The wire of the headset acts as an antenna, boosting fmtx transmission power over allowed limits. Therefore the daemon is monitoring plugged devices and powers the transmitter down, if the headset is not connected.

← Older revision		Revision as of 13:06, 18 August 2010
Line 7:		Line 7:


	[[Image:OMAP_architecture.png\|800px]]		[[Image:OMAP_architecture.png\|800px\|alt=Diagram of top-level multimedia architecture\|OMAP multimedia architecture]]


			Major challenge for the Multimedia framework is to provide the same interfaces like on desktop Linux for different hardware platforms. In order to achieve this objective, the whole audio processing is moved to the ARM side (ALSA / PulseAudio) and OpenMAX IL is introduced to abstract the hardware specific components (codecs and other hardware accelerated elements).
	Major challenge for the Multimedia framework is to provide the same interfaces like on desktop Linux for different hardware platforms. In order to achieve this objective, the whole audio processing is moved to the ARM side (ALSA / PulseAudio) and OpenMAX IL is introduced to abstract the hardware specific components (codecs and other hardware accelerated elements).

	==Video subsystem==		==Video subsystem==

	[[Image:mmf_video_decomposition.png]]		[[Image:mmf_video_decomposition.png\|frame\|center\|alt=UML diagram of video subsystem\|Video subsystem]]


	===Video Codecs===		===Video Codecs===

	Most of the video encoders and decoders are wrapped under the OpenMAX IL interface [OMAX], which abstracts the codec specific implementation (which in general can run on ARM or DSP). All the OpenMAX IL video codecs will be running on the DSP side in order to best exploit the HW acceleration provided by the OMAP 3430 platform. The DSP based OpenMAX IL components are loaded by TIs IL-Core, which in turn uses LCML and the DSP-Bridge. The ARM based OpenMAX IL are loaded via Bellagio IL-Core. Both IL-Cores are used by the gst-openmax bridge. The GStreamer framework resides on the ARM side.		Most of the video encoders and decoders are wrapped under the OpenMAX IL interface [OMAX], which abstracts the codec specific implementation (which in general can run on ARM or DSP). All the OpenMAX IL video codecs will be running on the DSP side in order to best exploit the HW acceleration provided by the OMAP 3430 platform. The DSP based OpenMAX IL components are loaded by TIs IL-Core, which in turn uses LCML and the DSP-Bridge. The ARM based OpenMAX IL are loaded via Bellagio IL-Core. Both IL-Cores are used by the gst-openmax bridge. The GStreamer framework resides on the ARM side.

	Video post processing is performed on DSS screen accelerator. DSS is used to do the colorspace conversion, the scaling and composition, including overlays. A separate external graphics accelerator is used to refresh the screen. In case of need (complicated use cases) the scaling and colorspace conversion can be done on the ARM side as well, but that is not recommened as it is not optimized.		Video post processing is performed on DSS screen accelerator. DSS is used to do the colorspace conversion, the scaling and composition, including overlays. A separate external graphics accelerator is used to refresh the screen. In case of need (complicated use cases) the scaling and colorspace conversion can be done on the ARM side as well, but that is not recommened as it is not optimized. A/V synchronization is done on the ARM, using an audio clock that is based on information from the audio interface.
	A/V synchronization is done on the ARM, using an audio clock that is based on information from the audio interface.

	The communication between ARM and DSP software is provided by the TI DSP bridge. Any messages or data buffers exchanged between the ARM and DSP go through it. This layer can be regarded as transparent from Multimedia Architecture point of view and hence it will not be described in this document.		The communication between ARM and DSP software is provided by the TI DSP bridge. Any messages or data buffers exchanged between the ARM and DSP go through it. This layer can be regarded as transparent from Multimedia Architecture point of view and hence it will not be described in this document.

	===GStreamer===		===GStreamer===

	GStreamer is a crossplatform media framework, covering most multimedia applications use cases from playback, to streaming and imaging. It is a huge collection of objects, interfaces, libraries and plugins. From the application point of view it is just one utility library that can be used by applications to process media streams. The library interface is actually a facade to a versatile collection of dynamic modules that implement the actual functionality. GStreamer core hides the complexity of timing issues, synchronization, buffering, threading, streaming and other functionalities that are needed to produce usable media application.		GStreamer is a crossplatform media framework, covering most multimedia applications use cases from playback, to streaming and imaging. It is a huge collection of objects, interfaces, libraries and plugins. From the application point of view it is just one utility library that can be used by applications to process media streams. The library interface is actually a facade to a versatile collection of dynamic modules that implement the actual functionality. GStreamer core hides the complexity of timing issues, synchronization, buffering, threading, streaming and other functionalities that are needed to produce usable media application.

Line 39:		Line 39:
	*GStreamer provides good modularity and flexibility. Hence, building applications on GStreamer in short time is possible.		*GStreamer provides good modularity and flexibility. Hence, building applications on GStreamer in short time is possible.
	*GStreamer is LGPL and it allows Multimedia Project to combine GStreamer with proprietary software.		*GStreamer is LGPL and it allows Multimedia Project to combine GStreamer with proprietary software.


	====Public interfaces provided by GStreamer====		====Public interfaces provided by GStreamer====
Line 46:		Line 45:
	! Interface name !! Description		! Interface name !! Description
	\|-		\|-
	\|GStreamer API\|\|Interface for Multimedia applications, VOIP etc		\| GStreamer API \|\| Interface for Multimedia applications, VOIP etc
	\|-		\|-
	\|playbin2\|\|Recommended high level element for playback.		\| <code>playbin2</code> \|\| Recommended high level element for playback.
	\|-		\|-
	\|uricodebin\|\|Recommended high level element for decoding.		\| <code>uricodebin</code> \|\| Recommended high level element for decoding.
	\|-		\|-
	\|tagreadbin\|\|Recommended high level element for fast metadata reading.		\| <code>tagreadbin</code> \|\| Recommended high level element for fast metadata reading.
	\|-		\|-
	\|camerabin\|\|Recommended high level element for camera application.		\| <code>camerabin</code> \|\| Recommended high level element for camera application.
	\|}		\|}

	===OpenMAX IL===		===OpenMAX IL===

	OpenMAX is an effort to provide an industry standard for a multimedia API. The standard defines 3 layers – OpenMAX DL (Development Layer), OpenMAX IL (Integration Layer) and OpenMAX AL (Application Layer). DL is a vendor specific and optional component. IL is the layer that interfaces with IL components (e.g. codecs). We will integrate TI IL-Core for the DSP components and Bellagio IL-Core for ARM components. Neither of the cores uses DL.		OpenMAX is an effort to provide an industry standard for a multimedia API. The standard defines 3 layers – OpenMAX DL (Development Layer), OpenMAX IL (Integration Layer) and OpenMAX AL (Application Layer). DL is a vendor specific and optional component. IL is the layer that interfaces with IL components (e.g. codecs). We will integrate TI IL-Core for the DSP components and Bellagio IL-Core for ARM components. Neither of the cores uses DL.
	We use GStreamer’s gomx module to transparently make OpenMAX IL components available to any GStreamer application.		We use GStreamer’s gomx module to transparently make OpenMAX IL components available to any GStreamer application.
Line 80:		Line 80:

	==Imaging subsystem==		==Imaging subsystem==

	The multimedia framework also provides support for imaging applications. The subsystem is illustrated below.		The multimedia framework also provides support for imaging applications. The subsystem is illustrated below.

	[[Image:Imaging_decomposition.png\|800px]]		[[Image:Imaging_decomposition.png\|800px\|alt=UML diagram of imaging subsystem\|Imaging subsystem]]


	===Camera Source===		===Camera Source===
	The GstV4L2CamSrc is a fork of the GstV4l2Src plugin. The reason for the fork is that the original plugin has lots of extra code for handing various V4L2 devices (such as tuner cards) and that made the code quite complex.
			The <code>GstV4L2CamSrc</code> is a fork of the <code>GstV4l2Src</code> plugin. The reason for the fork is that the original plugin has lots of extra code for handing various V4L2 devices (such as tuner cards) and that made the code quite complex.

	====GStreamer V4L2 Camera Source====		====GStreamer V4L2 Camera Source====
Line 120:		Line 122:
	! Interface name !! Description		! Interface name !! Description
	\|-		\|-
	\|V4L2 ioctl\|\|Extensions for V4L2 protocol		\| V4L2 ioctl \|\| Extensions for V4L2 protocol
	\|}		\|}

	==Audio Subsystem==		==Audio Subsystem==

	[[Image:Audio_decomposition.png]]		[[Image:Audio_decomposition.png\|frame\|center\|alt=UML diagram of audio subsystem\|audio subsystem]]


Line 224:		Line 226:
	MMF delivers parts of the notification mechanism. The subsytem is illustrated below:		MMF delivers parts of the notification mechanism. The subsytem is illustrated below:

	[[Image:Notification_decomposition.png]]		[[Image:Notification_decomposition.png\|frame\|center\|alt=UML diagram of notification subsystem\|Notification subsystem]]


Line 241:		Line 243:
	The multimedia framework supports the search engine when indexing media files. The subsystem is illustrated below:		The multimedia framework supports the search engine when indexing media files. The subsystem is illustrated below:

	[[Image:Metadata_decomposition.png]]		[[Image:Metadata_decomposition.png\|frame\|center\|alt=UML diagram of metadata subsystem\|Metadata subsystem]]


Line 252:		Line 254:
	It is neither user friendly nor always possible to run several multimedia use cases at the same time. To provide a predictable and stable behavior, multimedia components interact with a system wide policy component to keep concurrent multimedia use cases under control. The subsystem is illustrated below. The policy engine is based on the OHM framework. It is dynamically configurable with scripting and a prolog based rule database.		It is neither user friendly nor always possible to run several multimedia use cases at the same time. To provide a predictable and stable behavior, multimedia components interact with a system wide policy component to keep concurrent multimedia use cases under control. The subsystem is illustrated below. The policy engine is based on the OHM framework. It is dynamically configurable with scripting and a prolog based rule database.

	[[Image:Policy_decomposition.png]]		[[Image:Policy_decomposition.png\|frame\|center\|alt=UML diagram of policy subsystem\|Policy subsystem]]