plugins/compressor.h File Reference

The API to create runtime-loadable compression plugins. More...

#include <sys/types.h>

Include dependency graph for plugins/compressor.h:

Go to the source code of this file.

Classes

struct EqCompressorInfo

Information about one compressor. More...

Defines

Compressor Plugin API Versioning

#define EQ_COMPRESSOR_VERSION   1

The version of the Compressor API described by this header.

#define EQ_COMPRESSOR_VERSION_1   1

At least version 1 of the Compressor API is described by this header.

Compressor type name registry

The compressor type registry ensures the uniqueness of compressor names.
It is maintained by the Equalizer development team <info@equalizergraphics.com>. New types can be requested free of charge.

#define EQ_COMPRESSOR_NONE   0x1u

No Compressor.

#define EQ_COMPRESSOR_RLE_UNSIGNED   0x2u

RLE Compression of 4-byte tokens.

#define EQ_COMPRESSOR_RLE_BYTE   0x3u

RLE Compression of 1-byte tokens.

#define EQ_COMPRESSOR_RLE_3_BYTE   0x4u

RLE Compression of three 1-byte tokens.

#define EQ_COMPRESSOR_RLE_4_BYTE   0x5u

RLE Compression of four 1-byte tokens.

#define EQ_COMPRESSOR_RLE_4_FLOAT   0x6u

RLE Compression of four float32 tokens.

#define EQ_COMPRESSOR_RLE_4_HALF_FLOAT   0x7u

RLE Compression of four float16 tokens.

#define EQ_COMPRESSOR_DIFF_RLE_3_BYTE   0x8u

Differential RLE Compression of three 1-byte tokens.

#define EQ_COMPRESSOR_DIFF_RLE_4_BYTE   0x9u

Differential RLE Compression of three 1-byte tokens.

#define EQ_COMPRESSOR_RLE_4_BYTE_UNSIGNED   0xau

RLE Compression of one 4-byte token.

#define EQ_COMPRESSOR_PRIVATE   0xefffffffu

Private types -FOR DEVELOPMENT ONLY-.

Compressor token types

The compressor token type is reported by the DSO, and defines which type of input data can be processed by the given compressor.
It is used by Equalizer to select candidates for compression.

#define EQ_COMPRESSOR_DATATYPE_BYTE   1

Data is processed in one-byte tokens.

#define EQ_COMPRESSOR_DATATYPE_UNSIGNED   2

Data is processed in four-byte tokens.

#define EQ_COMPRESSOR_DATATYPE_HALF_FLOAT   3

Data is processed in float16 tokens.

#define EQ_COMPRESSOR_DATATYPE_FLOAT   4

Data is processed in float32 tokens.

#define EQ_COMPRESSOR_DATATYPE_3_BYTE   1024

Data is processed in three interleaved streams of one-byte tokens.

#define EQ_COMPRESSOR_DATATYPE_4_BYTE   1025

Data is processed in four interleaved streams of one-byte tokens.

#define EQ_COMPRESSOR_DATATYPE_3_HALF_FLOAT   1026

Data is processed in four interleaved streams of float16 tokens.

#define EQ_COMPRESSOR_DATATYPE_4_HALF_FLOAT   1027

Data is processed in four interleaved streams of float16 tokens.

#define EQ_COMPRESSOR_DATATYPE_3_FLOAT   1028

Data is processed in four interleaved streams of float32 tokens.

#define EQ_COMPRESSOR_DATATYPE_4_FLOAT   1029

Data is processed in four interleaved streams of float32 tokens.

#define EQ_COMPRESSOR_DATATYPE_3BYTE_1BYTE   2048

Data is processed in two interleaved streams, one 24 bit and one 8 bit.

Compressor capability flags

Capability flags define what special features a compressor supports.
They are queried from the DSO, and passed as input to certain functions to select a given mode.

#define EQ_COMPRESSOR_INPLACE   1

The compressor can (query time) or should (compress) write the compressed data in the same place as the uncompressed data.

#define EQ_COMPRESSOR_DATA_1D   2

The compressor can handle linear data (query time), or the input data is linear (compress, decompress).

#define EQ_COMPRESSOR_DATA_2D   4

The compressor can handle two-dimensional data (query time), or the input data is two-dimensional (compress, decompress).

#define EQ_COMPRESSOR_IGNORE_MSE   8

The compressor can (query time) or should (compress) ignore the most-significant element of the input data.

Functions

DSO information interface.

EQ_PLUGIN_API size_t EqCompressorGetNumCompressors ()

EQ_PLUGIN_API void EqCompressorGetInfo (const size_t n, EqCompressorInfo *const info)

Query information of the nth compressor in the DSO.

Compressor lifecycle management.

EQ_PLUGIN_API void * EqCompressorNewCompressor (const unsigned name)

Instantiate a new compressor.

EQ_PLUGIN_API void EqCompressorDeleteCompressor (void *const compressor)

Release a compressor instance.

EQ_PLUGIN_API void * EqCompressorNewDecompressor (const unsigned name)

Instantiate a new decompressor.

EQ_PLUGIN_API void EqCompressorDeleteDecompressor (void *const decompressor)

Release a decompressor instance.

Compressor worker functions

EQ_PLUGIN_API void EqCompressorCompress (void *const compressor, const unsigned name, void *const in, const eq_uint64_t *inDims, const eq_uint64_t flags)

Compress data.

EQ_PLUGIN_API unsigned EqCompressorGetNumResults (void *const compressor, const unsigned name)

Return the number of results produced by the last compression.

EQ_PLUGIN_API void EqCompressorGetResult (void *const compressor, const unsigned name, const unsigned i, void **const out, eq_uint64_t *const outSize)

Return the ith result of the last compression.

EQ_PLUGIN_API void EqCompressorDecompress (void *const decompressor, const unsigned name, const void *const *in, const eq_uint64_t *const inSizes, const unsigned numInputs, void *const out, eq_uint64_t *const outDims, const eq_uint64_t flags)

Decompress data.

Detailed Description

The API to create runtime-loadable compression plugins.

To implement a compression plugin, the following steps are to be taken:

Create a new shared library named EqualizerCompressorNAME.dll (Win32), libeqCompressorNAME.dylib (Mac OS X) or libeqCompressorNAME.so (Linux).
Define EQ_PLUGIN_API and then include eq/plugins/compressor.h (this header file).
Implement all C functions from this header file. You can use the default Equalizer compressors in src/lib/compressor as a template.
Put the library in the plugin search path (see eq::Global::getPluginDirectories(), defaults to EQ_PLUGIN_PATH or "/usr/local/share/Equalizer/plugins;.eqPlugins;$LD_LIBRARY_PATH".
Run the image unit test (tests/image) to verify your plugin.
Set the compression ratio and speed according to the output of the image unit test. Use the Equalizer RLE compressor as baseline.
Request official names for your compressors.

Definition in file plugins/compressor.h.

Define Documentation

#define EQ_COMPRESSOR_VERSION 1

The version of the Compressor API described by this header.

Definition at line 71 of file plugins/compressor.h.

#define EQ_COMPRESSOR_VERSION_1 1

At least version 1 of the Compressor API is described by this header.

Definition at line 73 of file plugins/compressor.h.

#define EQ_COMPRESSOR_RLE_UNSIGNED 0x2u

RLE Compression of 4-byte tokens.

Definition at line 87 of file plugins/compressor.h.

#define EQ_COMPRESSOR_RLE_BYTE 0x3u

RLE Compression of 1-byte tokens.

Definition at line 89 of file plugins/compressor.h.

#define EQ_COMPRESSOR_RLE_3_BYTE 0x4u

RLE Compression of three 1-byte tokens.

Definition at line 91 of file plugins/compressor.h.

#define EQ_COMPRESSOR_RLE_4_BYTE 0x5u

RLE Compression of four 1-byte tokens.

Definition at line 93 of file plugins/compressor.h.

#define EQ_COMPRESSOR_RLE_4_FLOAT 0x6u

RLE Compression of four float32 tokens.

Definition at line 95 of file plugins/compressor.h.

#define EQ_COMPRESSOR_RLE_4_HALF_FLOAT 0x7u

RLE Compression of four float16 tokens.

Definition at line 97 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DIFF_RLE_3_BYTE 0x8u

Differential RLE Compression of three 1-byte tokens.

Definition at line 99 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DIFF_RLE_4_BYTE 0x9u

Differential RLE Compression of three 1-byte tokens.

Definition at line 101 of file plugins/compressor.h.

#define EQ_COMPRESSOR_RLE_4_BYTE_UNSIGNED 0xau

RLE Compression of one 4-byte token.

Definition at line 103 of file plugins/compressor.h.

#define EQ_COMPRESSOR_PRIVATE 0xefffffffu

Private types -FOR DEVELOPMENT ONLY-.

Any name equal or bigger than this can be used for in-house development and testing. As soon as the Compressor DSO is distributed, request public types free of charge from info@equalizergraphics.com.

Definition at line 112 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_BYTE 1

Data is processed in one-byte tokens.

Definition at line 124 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_UNSIGNED 2

Data is processed in four-byte tokens.

Definition at line 126 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_HALF_FLOAT 3

Data is processed in float16 tokens.

Definition at line 128 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_FLOAT 4

Data is processed in float32 tokens.

Definition at line 130 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_3_BYTE 1024

Data is processed in three interleaved streams of one-byte tokens.

Definition at line 134 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_4_BYTE 1025

Data is processed in four interleaved streams of one-byte tokens.

Definition at line 136 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_3_HALF_FLOAT 1026

Data is processed in four interleaved streams of float16 tokens.

Definition at line 138 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_4_HALF_FLOAT 1027

Data is processed in four interleaved streams of float16 tokens.

Definition at line 140 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_3_FLOAT 1028

Data is processed in four interleaved streams of float32 tokens.

Definition at line 142 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_4_FLOAT 1029

Data is processed in four interleaved streams of float32 tokens.

Definition at line 144 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATATYPE_3BYTE_1BYTE 2048

Data is processed in two interleaved streams, one 24 bit and one 8 bit.

Definition at line 146 of file plugins/compressor.h.

#define EQ_COMPRESSOR_DATA_1D 2

The compressor can handle linear data (query time), or the input data is linear (compress, decompress).

Typically used for binary data.

Definition at line 166 of file plugins/compressor.h.

Referenced by EqCompressorCompress(), and EqCompressorDecompress().

#define EQ_COMPRESSOR_DATA_2D 4

The compressor can handle two-dimensional data (query time), or the input data is two-dimensional (compress, decompress).

Typically used for image data.

Definition at line 172 of file plugins/compressor.h.

Referenced by eq::Image::compressPixelData(), and eq::Image::setPixelData().

#define EQ_COMPRESSOR_IGNORE_MSE 8

The compressor can (query time) or should (compress) ignore the most-significant element of the input data.

Typically used for image data when the alpha-channel is present in the input data, but unneeded.

Definition at line 178 of file plugins/compressor.h.

Referenced by eq::Image::compressPixelData(), EqCompressorCompress(), EqCompressorDecompress(), and eq::Image::setPixelData().

Function Documentation

EQ_PLUGIN_API size_t EqCompressorGetNumCompressors ( )

Returns:: the number of compressors implemented in the DSO.

Definition at line 87 of file compressor/compressor.cpp.

EQ_PLUGIN_API void EqCompressorGetInfo	(	const size_t	n,
		EqCompressorInfo *const	info
	)

Query information of the nth compressor in the DSO.

Definition at line 93 of file compressor/compressor.cpp.

EQ_PLUGIN_API void* EqCompressorNewCompressor ( const unsigned name )

Instantiate a new compressor.

This function has to create a new instance of the given compressor type. Multiple instances might be used concurrently. One given instance is always used from one thread at any given time.

Parameters:

name

the type name of the compressor.

Returns:: an opaque pointer to the compressor instance.

Definition at line 98 of file compressor/compressor.cpp.

EQ_PLUGIN_API void EqCompressorDeleteCompressor ( void *const compressor )

Release a compressor instance.

Parameters:

compressor

the compressor instance to free.

Definition at line 105 of file compressor/compressor.cpp.

EQ_PLUGIN_API void* EqCompressorNewDecompressor ( const unsigned name )

Instantiate a new decompressor.

This function might create a new instance of the given decompressor type. Multiple instances might be used concurrently. One given instance is always used from one thread at any given time. State-less decompressors might return 0.

Parameters:

name

the type name of the decompressor.

Returns:: an opaque pointer to the decompressor instance, or 0 if no instance is needed by the implementation.

Definition at line 110 of file compressor/compressor.cpp.

EQ_PLUGIN_API void EqCompressorDeleteDecompressor ( void *const decompressor )

Release a decompressor instance.

Parameters:

decompressor

the decompressor instance to free.

Definition at line 115 of file compressor/compressor.cpp.

EQ_PLUGIN_API void EqCompressorCompress	(	void *const	compressor,
		const unsigned	name,
		void *const	in,
		const eq_uint64_t *	inDims,
		const eq_uint64_t	flags
	)

Compress data.

The number of dimensions in the input and output data is given as a flag. The input dimensions give an offset and a size for each dimension in the format dim0_offset, dim0_size, dim1_offset, ..., dimN_size. The offset does not apply to the input pointer, it is merely a hint on where the data is positioned, e.g., where a 2D image is positioned in a virtual framebuffer. The size of the input data is mul( inDims[1,3,...,n] ) * sizeof( info->dataType ).

The compressor has to store the results internally in its instance data The result of the compression run will be queried later. Results of previous compression do not have to be retained, i.e., they can be overwritten on subsequent compression runs.

Parameters:

	compressor	the compressor instance.
	name	the type name of the compressor.
	in	the pointer to the input data.
	inDims	the dimensions of the input data.
	flags	capability flags for the compression.

Definition at line 121 of file compressor/compressor.cpp.

References EQ_COMPRESSOR_DATA_1D, and EQ_COMPRESSOR_IGNORE_MSE.

EQ_PLUGIN_API unsigned EqCompressorGetNumResults	(	void *const	compressor,
		const unsigned	name
	)

Return the number of results produced by the last compression.

A compressor might generate multiple output stream, e.g., when operating on structured data or using parallel compression routines.

Parameters:

	compressor	the compressor instance.
	name	the type name of the compressor.

Returns:: the number of output results.

Definition at line 134 of file compressor/compressor.cpp.

EQ_PLUGIN_API void EqCompressorGetResult	(	void *const	compressor,
		const unsigned	name,
		const unsigned	i,
		void **const	out,
		eq_uint64_t *const	outSize
	)

Return the ith result of the last compression.

Parameters:

	compressor	the compressor instance.
	name	the type name of the compressor.
	i	the result index to return.
	out	the return value to store the result pointer.
	outSize	the return value to store the result size in bytes.

Definition at line 142 of file compressor/compressor.cpp.

References eq::base::Buffer< T >::getData(), eq::base::Buffer< T >::getMaxSize(), and eq::base::Buffer< T >::getSize().

Here is the call graph for this function:

EQ_PLUGIN_API void EqCompressorDecompress	(	void *const	decompressor,
		const unsigned	name,
		const void const	in,
		const eq_uint64_t *const	inSizes,
		const unsigned	numInputs,
		void *const	out,
		eq_uint64_t *const	outDims,
		const eq_uint64_t	flags
	)

Decompress data.

The decompressor gets all result pointers as produced by the compressor as input. The routine should use the output buffer fully. For dimensions and output size see EqCompressorCompress.

Parameters:

	decompressor	the decompressor instance, can be 0.
	name	the type name of the decompressor.
	in	the pointer to an array of input data pointers.
	inSizes	the array of input data sizes in bytes.
	numInputs	the number of input data elements.
	out	the pointer to a pre-allocated buffer for the uncompressed output result.
	outDims	the dimensions of the output data.
	flags	capability flags for the decompression.

See also:: EqCompressorCompress

Definition at line 156 of file compressor/compressor.cpp.

References EQ_COMPRESSOR_DATA_1D, and EQ_COMPRESSOR_IGNORE_MSE.


Classes
struct	EqCompressorInfo
	Information about one compressor. More...
Defines
Compressor Plugin API Versioning
#define	EQ_COMPRESSOR_VERSION 1
	The version of the Compressor API described by this header.
#define	EQ_COMPRESSOR_VERSION_1 1
	At least version 1 of the Compressor API is described by this header.
Compressor type name registry
The compressor type registry ensures the uniqueness of compressor names. It is maintained by the Equalizer development team <info@equalizergraphics.com>. New types can be requested free of charge.
#define	EQ_COMPRESSOR_NONE 0x1u
	No Compressor.
#define	EQ_COMPRESSOR_RLE_UNSIGNED 0x2u
	RLE Compression of 4-byte tokens.
#define	EQ_COMPRESSOR_RLE_BYTE 0x3u
	RLE Compression of 1-byte tokens.
#define	EQ_COMPRESSOR_RLE_3_BYTE 0x4u
	RLE Compression of three 1-byte tokens.
#define	EQ_COMPRESSOR_RLE_4_BYTE 0x5u
	RLE Compression of four 1-byte tokens.
#define	EQ_COMPRESSOR_RLE_4_FLOAT 0x6u
	RLE Compression of four float32 tokens.
#define	EQ_COMPRESSOR_RLE_4_HALF_FLOAT 0x7u
	RLE Compression of four float16 tokens.
#define	EQ_COMPRESSOR_DIFF_RLE_3_BYTE 0x8u
	Differential RLE Compression of three 1-byte tokens.
#define	EQ_COMPRESSOR_DIFF_RLE_4_BYTE 0x9u
	Differential RLE Compression of three 1-byte tokens.
#define	EQ_COMPRESSOR_RLE_4_BYTE_UNSIGNED 0xau
	RLE Compression of one 4-byte token.
#define	EQ_COMPRESSOR_PRIVATE 0xefffffffu
	Private types -FOR DEVELOPMENT ONLY-.
Compressor token types
The compressor token type is reported by the DSO, and defines which type of input data can be processed by the given compressor. It is used by Equalizer to select candidates for compression.
#define	EQ_COMPRESSOR_DATATYPE_BYTE 1
	Data is processed in one-byte tokens.
#define	EQ_COMPRESSOR_DATATYPE_UNSIGNED 2
	Data is processed in four-byte tokens.
#define	EQ_COMPRESSOR_DATATYPE_HALF_FLOAT 3
	Data is processed in float16 tokens.
#define	EQ_COMPRESSOR_DATATYPE_FLOAT 4
	Data is processed in float32 tokens.
#define	EQ_COMPRESSOR_DATATYPE_3_BYTE 1024
	Data is processed in three interleaved streams of one-byte tokens.
#define	EQ_COMPRESSOR_DATATYPE_4_BYTE 1025
	Data is processed in four interleaved streams of one-byte tokens.
#define	EQ_COMPRESSOR_DATATYPE_3_HALF_FLOAT 1026
	Data is processed in four interleaved streams of float16 tokens.
#define	EQ_COMPRESSOR_DATATYPE_4_HALF_FLOAT 1027
	Data is processed in four interleaved streams of float16 tokens.
#define	EQ_COMPRESSOR_DATATYPE_3_FLOAT 1028
	Data is processed in four interleaved streams of float32 tokens.
#define	EQ_COMPRESSOR_DATATYPE_4_FLOAT 1029
	Data is processed in four interleaved streams of float32 tokens.
#define	EQ_COMPRESSOR_DATATYPE_3BYTE_1BYTE 2048
	Data is processed in two interleaved streams, one 24 bit and one 8 bit.
Compressor capability flags
Capability flags define what special features a compressor supports. They are queried from the DSO, and passed as input to certain functions to select a given mode.
#define	EQ_COMPRESSOR_INPLACE 1
	The compressor can (query time) or should (compress) write the compressed data in the same place as the uncompressed data.
#define	EQ_COMPRESSOR_DATA_1D 2
	The compressor can handle linear data (query time), or the input data is linear (compress, decompress).
#define	EQ_COMPRESSOR_DATA_2D 4
	The compressor can handle two-dimensional data (query time), or the input data is two-dimensional (compress, decompress).
#define	EQ_COMPRESSOR_IGNORE_MSE 8
	The compressor can (query time) or should (compress) ignore the most-significant element of the input data.
Functions
DSO information interface.
EQ_PLUGIN_API size_t	EqCompressorGetNumCompressors ()
EQ_PLUGIN_API void	EqCompressorGetInfo (const size_t n, EqCompressorInfo *const info)
	Query information of the nth compressor in the DSO.
Compressor lifecycle management.
EQ_PLUGIN_API void *	EqCompressorNewCompressor (const unsigned name)
	Instantiate a new compressor.
EQ_PLUGIN_API void	EqCompressorDeleteCompressor (void *const compressor)
	Release a compressor instance.
EQ_PLUGIN_API void *	EqCompressorNewDecompressor (const unsigned name)
	Instantiate a new decompressor.
EQ_PLUGIN_API void	EqCompressorDeleteDecompressor (void *const decompressor)
	Release a decompressor instance.
Compressor worker functions
EQ_PLUGIN_API void	EqCompressorCompress (void const compressor, const unsigned name, void const in, const eq_uint64_t *inDims, const eq_uint64_t flags)
	Compress data.
EQ_PLUGIN_API unsigned	EqCompressorGetNumResults (void *const compressor, const unsigned name)
	Return the number of results produced by the last compression.
EQ_PLUGIN_API void	EqCompressorGetResult (void const compressor, const unsigned name, const unsigned i, void const out, eq_uint64_t const outSize)
	Return the ith result of the last compression.
EQ_PLUGIN_API void	EqCompressorDecompress (void const decompressor, const unsigned name, const void const in, const eq_uint64_t const inSizes, const unsigned numInputs, void const out, eq_uint64_t const outDims, const eq_uint64_t flags)
	Decompress data.