Pike Manual

Module Gz

Description

The Gz module contains functions to compress and uncompress strings using the same algorithm as the program gzip. Compressing can be done in streaming mode or all at once.

The Gz module consists of two classes; Gz.deflate and Gz.inflate. Gz.deflate is used to pack data and Gz.inflate is used to unpack data. (Think "inflatable boat")

Note

Note that this module is only available if the gzip library was available when Pike was compiled.

Note that although these functions use the same algorithm as gzip, they do not use the exact same format, so you cannot directly unzip gzipped files with these routines. Support for this will be added in the future.

Inherit _Gz: inherit _Gz : _Gz

Method check_header: string(8bit)|zerocheck_header(Stdio.Stream|voidf, Stdio.Buffer|string(8bit)|voidbuf)
Description: Check whether a file has a valid gzip header.
Parameter f: File to check.
Parameter buf: Prefix of f.
Returns: Returns the content of f after the gzip header if a header was found. Returns 0 (zero) if there was no header.

Method make_header: intmake_header(Stdio.Stream|Stdio.Bufferf)
Description: Write a gzip header to a file or buffer.
Parameter f: File or buffer to write a gzip header to.
Returns: Returns 1 on success and 0 (zero) on failure.

Class Gz.File

Inherit _file: inherit ._file : _file
Description: Allows the user to open a Gzip archive and read and write it's contents in an uncompressed form, emulating the Stdio.File interface.
Note: An important limitation on this class is that it may only be used for reading or writing, not both at the same time. Please also note that if you want to reopen a file for reading after a write, you must close the file before calling open or strange effects might be the result.

Method create: Gz.FileGz.File(void|string|int|Stdio.Streamfile, void|stringmode)
Parameter file: Filename or filedescriptor of the gzip file to open, or an already open Stream.
Parameter mode: mode for the file. Defaults to "rb".
See also: openStdio.File

Method line_iterator: String.SplitIterator|Stdio.LineIteratorline_iterator(int|voidtrim)
Description: Returns an iterator that will loop over the lines in this file. If trim is true, all '\r' characters will be removed from the input.

Method open

intopen(string|int|Stdio.Streamfile, void|stringmode)

Parameter file

Filename or filedescriptor of the gzip file to open, or an already open Stream.

Parameter mode

mode for the file. Defaults to "rb". May be one of the following:

rb: read mode
wb: write mode
ab: append mode

For the wb and ab mode, additional parameters may be specified. Please se zlib manual for more info.

Returns

non-zero if successful.

Method read: int|stringread(void|intlength)
Description: Reads data from the file. If no argument is given, the whole file is read.

Method read_function: function(:string) read_function(intnbytes)
Description: Returns a function that when called will call read with nbytes as argument. Can be used to get various callback functions, eg for the fourth argument to String.SplitIterator.

Class Gz._file

Description: Low-level implementation of read/write support for GZip files

Method close: intclose()
Description: closes the file
Returns: 1 if successful

Method create: Gz._fileGz._file(void|string|Stdio.StreamgzFile, void|stringmode)
Description: Opens a gzip file for reading.

Method eof: booleof()
Returns: 1 if EOF has been reached.

Method open

intopen(string|int|Stdio.Streamfile, void|stringmode)

Description

Opens a file for I/O.

Parameter file

The filename or an open filedescriptor or Stream for the GZip file to use.

Parameter mode

Mode for the file operations. Defaults to read only. The following mode characters are unique to Gz.File.

`"0"`	Values 0 to 9 set the compression level from no compression to maximum available compression. Defaults to 6.
`"1"`
`"2"`
`"3"`
`"4"`
`"5"`
`"6"`
`"7"`
`"8"`
`"9"`
`"f"`	Sets the compression strategy to `FILTERED`.
`"h"`	Sets the compression strategy to `HUFFMAN_ONLY`.

Note

If the object already has been opened, it will first be closed.

Method read: int|stringread(intlen)
Description: Reads len (uncompressed) bytes from the file. If read is unsuccessful, 0 is returned.

Method seek: intseek(intpos, void|inttype)
Description: Seeks within the file.
Parameter pos: Position relative to the searchtype.
Parameter type: SEEK_SET = set current position in file to pos SEEK_CUR = new position is current+pos SEEK_END is not supported.
Returns: New position or negative number if seek failed.

Method setparams: intsetparams(void|int(0..9)level, void|intstrategy, void|int(8..15)window_size)
Description: Sets the encoding level, strategy and window_size.
See also: Gz.deflate

Method tell: inttell()
Returns: the current position within the file.

Method write: intwrite(stringdata)
Description: Writes the data to the file.
Returns: the number of bytes written to the file.

Module Bz2

Description

The Bz2 module contains functions to compress and uncompress strings using the same algorithm as the program bzip2. Compressing and decompressing can be done in streaming mode feeding the compress and decompress objects with arbitrarily large pieces of data.

The Bz2 module consists of three classes; Bz2.Deflate, Bz2.Inflate and Bz2.File. Bz2.Deflate is used to compress data and Bz2.Inflate is used to uncompress data. Bz2.File is used to handle Bzip2 files.

Note

Note that this module is only available if libbzip2 was available when Pike was compiled.

Note that although the functions in Inflate and Deflate use the same algorithm as bzip2, they do not use the exact same format, so you can not directly zip files or unzip zip-files using those functions. That is why there exists a third class for files.

Inherit Bz2: inherit "___Bz2" : Bz2

Class Bz2.File

Inherit File: inherit Bz2::File : File

Method line_iterator: String.SplitIterator|Stdio.LineIteratorline_iterator(int|voidtrim)
Description: Returns an iterator that will loop over the lines in this file. If trim is true, all '\r' characters will be removed from the input.

Method read_function: function(:string) read_function(intnbytes)
Description: Returns a function that when called will call read with nbytes as argument. Can be used to get various callback functions, eg for the fourth argument to String.SplitIterator.

Module HPack

Description

Implementation of the HPACK (RFC 7541) header packing standard.

This is the header packing system that is used in HTTP/2 (RFC 7540).

Inherit "___HPack": inherit "___HPack" : "___HPack"

Constant DEFAULT_HEADER_TABLE_SIZE

constantint HPack.DEFAULT_HEADER_TABLE_SIZE

Description

This is the default static maximum size of the dynamic header table.

This constant is taken from RFC 7540 section 6.5.2.

Constant static_header_tab

constant HPack.static_header_tab

Description

Table of static headers. RFC 7541 appendix A, Table 1.

Array

array(string(8bit)) 0..60

Array
`string(8bit)0`	Header name.
`string(8bit)1`	Default value.

Note

Note that this table is indexed starting on 0 (zero), while the corresponding table in RFC 7541 starts on 1 (one).

Variable static_header_index

protectedmapping(string(8bit):int|mapping(string(8bit):int)) HPack.static_header_index

Description

Index for static_header_tab.

Note

Note that the indices are offset by 1 (one).

Note

This variable should be regarded as a constant.

This variable is used to initialize the header index in the Context.

See also

static_header_tab, Context()->header_index

Method create_index: protectedmapping(string(8bit):int|mapping(string(8bit):int)) create_index(array(array(string(8bit))) tab)
Description: Helper function used to create the static_header_index.

Method update_index: protectedvoidupdate_index(mapping(string(8bit):int|mapping(string(8bit):int)) index, inti, array(string(8bit)) key)
Description: Update the specified encoder lookup index.
Parameter index: Lookup index to add an entry to.
Parameter key: Lookup key to add.
Parameter i: Value to store in the index for the key.

Enum HPack.HPackFlags

Description: Flags for Context()->encode_header() et al.

Constant HEADER_INDEXED: constant HPack.HEADER_INDEXED
Description: Indexed header.

Constant HEADER_INDEXED_MASK: constant HPack.HEADER_INDEXED_MASK
Description: Bitmask for indexing mode.

Constant HEADER_NEVER_INDEXED: constant HPack.HEADER_NEVER_INDEXED
Description: Never indexed header.

Constant HEADER_NOT_INDEXED: constant HPack.HEADER_NOT_INDEXED
Description: Unindexed header.

Class HPack.Context

Description

Context for an HPack encoder or decoder.

This class implements the majority of RFC 7541.

Functions of interest are typically encode() and decode().

Variable dynamic_headers

protectedarray(array(string(8bit))) HPack.Context.dynamic_headers

Description

Table of currently available dynamically defined headers.

New entries are appended last, and the first dynamic_prefix elements are not used.

See also

header_index, add_header()

Variable dynamic_max_size: protectedint HPack.Context.dynamic_max_size
Description: Current upper size limit in bytes for dynamic_headers.
See also: set_dynamic_size()

Variable dynamic_prefix: protectedint HPack.Context.dynamic_prefix
Description: Index of first avaiable header in dynamic_headers.

Variable dynamic_size: protectedint HPack.Context.dynamic_size
Description: Current size in bytes of dynamic_headers.

Variable header_index

protectedmapping(string(8bit):int|mapping(string(8bit):int)) HPack.Context.header_index

Description

Index into dynamic_headers and static_header_tab.

"header_name" : mapping(string(8bit):int)|int

Indexed on the header name in lower-case. The value is one of:

int

Index value for the header value "".

mapping(string(8bit):int)

"header_value" : mapping(string(8bit):int)

Index value for the corresponding header value.

The index values in turn are coded as follows:

`(1..)`	Index into `static_header_tab` offset by `1`.
`0`	Not used.
`(..-1)`	Inverted (`~()) index into `dynamic_headers`.

See also

dynamic_headers, static_header_tab, add_header()

Variable static_max_size: protectedint HPack.Context.static_max_size
Description: Static upper size limit in bytes for dynamic_headers.
See also: create(), set_dynamic_size()

Method add_header: int(0)|int(62)add_header(string(8bit)header, string(8bit)value)
Description: Add a header to the table of known headers and to the header index.
Parameter header: Name of header to add.
Parameter value: Value of the header.
Returns: Returns 0 (zero) if the header was too large to store. Returns the encoding key for the header on success (this is always sizeof(static_header_tab + 1 (ie 62), as new headers are prepended to the dynamic header table.
Note: Adding a header may cause old headers to be evicted from the table.
See also: get_indexed_header()

Method create: HPack.ContextHPack.Context(int|voidprotocol_dynamic_max_size)
Description: Create a new HPack Context.
Parameter static_max_size: This is the static maximum size in bytes (as calculated by RFC 7541 section 4.1) of the dynamic header table. It defaults to DEFAULT_HEADER_TABLE_SIZE, and is the upper limit for set_dynamic_size().
See also: set_dynamic_size()

Method decode: array(array(string(8bit)|HPackFlags)) decode(Stdio.Bufferbuf)
Description: Decode a HPack header block.
Parameter buf: Input buffer.
Returns: Returns an array of headers. Cf decode_header().
See also: decode_header(), encode()

Method decode_header

array(string(8bit)|HPackFlags) decode_header(Stdio.Bufferbuf)

Description

Decode a single HPack header.

Parameter buf

Input buffer.

Returns

Returns UNDEFINED on empty buffer. Returns an array with a header and value otherwise:

Array
`string(8bit)0`	Name of the header. Under normal circumstances this is always lower-case, but no check is currently performed.
`string(8bit)1`	Value of the header.
`HPackFlags\|void2`	Optional encoding flags. Only set for fields having `HEADER_NEVER_INDEXED`.

The elements in the array are in the same order and compatible with the arguments to encode_header().

Throws

Throws on encoding errors.

Note

The returned array MUST NOT be modified.

Note

In future implementations the result array may get extended with a flag field.

Note

The in-band signalling of encoding table sizes is handled internally.

See also

decode(), encode_header()

Method encode: voidencode(array(array(string(8bit)|HPackFlags)) headers, Stdio.Bufferbuf)
Description: Encode a full set of headers.
Parameter headers: An array of ({ header, value })-tuples.
Parameter buf: Output buffer.
See also: encode_header(), decode()

Method encode: variantstring(8bit)encode(array(array(string(8bit))) headers)
Description: Convenience variant of encode().
Parameter headers: An array of ({ header, value })-tuples.
Returns: Returns the corresponding HPack encoding.

Method encode_header: voidencode_header(Stdio.Bufferbuf, string(8bit)header, string(8bit)value, HPackFlags|voidflags)
Description: Encode a single HPack header.
Parameter buf: Output buffer.
Parameter header: Name of header. This should under normal circumstances be a lower-case string, but this is currently not checked.
Parameter value: Header value.
Parameter flags: Optional encoding flags.
See also: encode(), decode_header()

Method evict_dynamic_headers: protectedvoidevict_dynamic_headers()
Description: Evict dynamic headers until dynamic_size goes below dynamic_max_size.

Method get_indexed_header

array(string(8bit)) get_indexed_header(int(1..)index)

Description

Lookup a known header.

Parameter index

Encoding key for the header to retrieve.

Returns

Returns UNDEFINED on unknown header. Returns an array with a header and value otherwise:

Array
`string(8bit)0`	Name of the header. Under normal circumstances this is always lower-case, but no check is currently performed.
`string(8bit)1`	Value of the header.

See also

add_header()

Method put_int: protectedvoidput_int(Stdio.Bufferbuf, int(8bit)bits, int(8bit)mask, intvalue)
Description: Encode an integer with the HPack integer encoding.
Parameter buf: Output buffer.
Parameter bits: Bits that should always be set in the first byte of output.
Parameter mask: Bitmask for the value part of the first byte of output.
Parameter value: Integer value to encode.

Method put_string

protectedvoidput_string(Stdio.Bufferbuf, string(8bit)str)

Description

Encode a string with the HPack string encoding.

Parameter buf

Output buffer.

Parameter str

String to output.

The encoder will huffman_encode() the string if that renders a shorter encoding than the verbatim string.

Method set_dynamic_size: voidset_dynamic_size(Stdio.Bufferbuf, int(0..)new_max_size)
Description: Set the dynamic maximum size of the dynamic header lookup table.
Parameter buf: Output buffer.
Parameter new_max_size: New dynamic maximum size in bytes (as calculated by RFC 7541 section 4.1).
Note: This function can be used to clear the dynamic header table by setting the size to zero.
Note: Also note that the new_max_size has an upper bound that is limited by static_max_size.
See also: encode_header(), encode(), create().

16. Compression

Module Gz

Class Gz.File

Class Gz._file

Module Bz2

Class Bz2.File

Module HPack

Enum HPack.HPackFlags

Class HPack.Context