kZstdStream Class Reference

Description

Implements the Zstandard format for compression and decompression.

The kZstdStream class implements the Zstandard (zstd) compression algorithm. Data is compressed as it is written and/or decompressed as it is read. The parsing/formatting algorithms implemented by kZstdStream are compatible with Zstandard-aware compression utilities such as zstd on Linux and 7zip-zstd (a custom build of 7zip that supports newer codecs) on Windows.

The zstd algorithm offers a wide range of speeds and densities through 22 compression presets. Lower presets are generally faster, while higher presets offer better compression density. The kZstdPreset enumeration defines numeric limits (i.e., kZSTD_PRESET_MIN, kZSTD_PRESET_MAX) and recommended values for compression presets (i.e., kZSTD_PRESET_FAST, kZSTD_PRESET_DEFAULT, kZSTD_PRESET_DENSE).

The amount of memory required for compression varies considerably with the preset, as summarized in the following table.

Preset	Compression	Decompression
1	1.4 MB	0.3 MB
3	2.6 MB	0.3 MB
5	3.4 MB	0.3 MB
7	6.0 MB	0.3 MB
9	9.0 MB	0.3 MB
11	24.5 MB	0.3 MB
13	29.5 MB	0.3 MB
17	50.0 MB	0.3 MB
19	59.0 MB	0.3 MB
20	198 MB	0.3 MB
22	788 MB	0.3 MB

The examples below illustrate the use of kZstdStream. The output of the WriteCompressedObject sample function would be a ".kdat6.zst" file – a serialized kdat6 object, encapsulated in a compressed zst file. This stream-layering approach can be used to compress and decompress arbitrary content. (An alternative to the stream-layering approach is discussed after these examples.)

kStatus WriteCompressedObject(kObject source, const kChar* path)
{
    kFile file = kNULL;
    kZstdStream zstd = kNULL;
    kSerializer writer = kNULL;

    kTry
    {
        //open a file for writing
        kTest(kFile_Construct(&file, path, kFILE_MODE_WRITE, kNULL));

        //construct a kZstdStream object; use fast compression
        kTest(kZstdStream_Construct(&zstd, file, kZSTD_MODE_WRITE, kZSTD_PRESET_FAST, kNULL));

        //construct a kDat6Serializer object
        kTest(kDat6Serializer_Construct(&writer, zstd, kNULL));

        //serialize the object to file
        kTest(kSerializer_WriteObject(writer, source));
    }
    kFinally
    {
        //tear down in correct order
        kObject_Destroy(writer);
        kObject_Destroy(zstd);
        kObject_Destroy(file);

        kEndFinally();
    }

    return kOK;
}

kStatus ReadCompressedObject(kObject *dest, const kChar* path)
{
    kFile file = kNULL;
    kZstdStream zstd = kNULL;
    kSerializer reader = kNULL;

    kTry
    {
        //open a file for writing
        kTest(kFile_Construct(&file, path, kFILE_MODE_READ, kNULL));

        //construct a kZstdStream object
        kTest(kZstdStream_Construct(&zstd, file, kZSTD_MODE_READ, 0, kNULL));

        //construct a kDat6Serializer object
        kTest(kDat6Serializer_Construct(&reader, zstd, kNULL));

        //deserialize the object from file
        kTest(kSerializer_ReadObject(reader, dest, kNULL));
    }
    kFinally
    {
        //tear down in correct order
        kObject_Destroy(reader);
        kObject_Destroy(zstd);
        kObject_Destroy(file);

        kEndFinally();
    }

    return kOK;
}

While the stream-layering approach is flexible and efficient, it requires the reader and writer to coordinate on the compression approach. To simplify usage, the kDat6Serializer class offers built-in support for compression algorithms present in the FireSync platform. A version of the WriteCompressedObject sample function, revised to make use of kDat6Serializer's built in compression support, is shown below. The output from this revised example is a "kdat6" file – a serialized kdat6 object that, internally, contains compressed content.

kStatus WriteCompressedObject(kObject source, const kChar* path)
{
    kFile file = kNULL;
    kSerializer writer = kNULL;

    kTry
    {
        //open a file for writing
        kTest(kFile_Construct(&file, path, kFILE_MODE_WRITE, kNULL));

        //construct a kDat6Serializer object
        kTest(kDat6Serializer_Construct(&writer, file, kNULL));

        //configure compression; note, generic platform compression presets can be used here
        kTest(kDat6Serializer_EnableCompression(writer, kCOMPRESSION_TYPE_ZSTD, kCOMPRESSION_PRESET_DEFAULT));

        //serialize the object to file
        kTest(kSerializer_WriteObject(writer, source));
    }
    kFinally
    {
        //tear down in correct order
        kObject_Destroy(writer);
        kObject_Destroy(file);

        kEndFinally();
    }

    return kOK;
}

kStatus ReadCompressedObject(kObject *dest, const kChar* path)
{
    kFile file = kNULL;
    kSerializer reader = kNULL;

    kTry
    {
        //open a file for writing
        kTest(kFile_Construct(&file, path, kFILE_MODE_READ, kNULL));

        //construct a kDat6Serializer object; no need to specify decompression algorithm
        kTest(kDat6Serializer_Construct(&reader, file, kNULL));

        //deserialize the object from file
        kTest(kSerializer_ReadObject(reader, dest, kNULL));
    }
    kFinally
    {
        //tear down in correct order
        kObject_Destroy(reader);
        kObject_Destroy(file);

        kEndFinally();
    }

    return kOK;
}

For applications that only need to write a single compressed object to a "kdat6" file, the kDat6Serializer class provides the kDat6Serializer_SaveCompressed method, which can be used to implement the example above in a single line of code. Content written using kDat6Serializer_SaveCompressed can be read back using standard platform utility methods such as kLoad6.

Note that the layered-streams approach will usually produce more dense/compact compression because it enables the algorithm to make use of context across objects. In contrast, when using the built-in compression features in kDat6Serializer, each object is compressed individually. If the source objects are large, there will be little difference in compression density between the two approaches. But if the source objects are small, the layered-streams approach may yield better results.

The kZstdStream class is available only on host platforms (i.e., Windows, Linux). If support is desired on embedded platforms, please contact FSS to discuss requirements.

Inheritance diagram for kZstdStream:

Public Member Functions
kStatus	kZstdStream_Construct (kZstdStream *zStream, kStream stream, kZstdMode mode, k32s level, kAlloc allocator)
	Constructs a kZstdStream object. More...
kStatus	kZstdStream_FinishRead (kZstdStream zStream)
	Finishes reading the current Zstandard compression frame. More...
kStatus	kZstdStream_FinishWrite (kZstdStream zStream)
	Finishes writing the current Zstandard compression frame. More...

---

The documentation for this class was generated from the following file:

• Utils/kZstdStream.h