camel_tools.utils.normalize

This submodule contains functions for normalizing Arabic text in different encodings. See Encoding Schemes for more information on encodings.

Functions

camel_tools.utils.normalize.normalize_unicode(s, compatibility=True)

Normalize Unicode strings into their canonically composed form or (i.e. characters that can be written as a combination of unicode characters are converted to their single character form).

Note: This is essentially a call to unicodedata.normalize() with form ‘NFC’ if compatibility is False or ‘NFKC’ if it’s True.

Parameters:

s (str) – The string to be normalized.
compatibility (bool, optional) – Apply compatibility decomposition. Defaults to True.

Returns:

The normalized string.

Return type:

str

camel_tools.utils.normalize.normalize_alef_maksura_ar(s)

Normalize all occurences of Alef Maksura characters to a Yeh character in an Arabic string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_maksura_bw(s)

Normalize all occurences of Alef Maksura characters to a Yeh character in a Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_maksura_safebw(s)

Normalize all occurences of Alef Maksura characters to a Yeh character in a Safe Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_maksura_xmlbw(s)

Normalize all occurences of Alef Maksura characters to a Yeh character in a XML Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_maksura_hsb(s)

Normalize all occurences of Alef Maksura characters to a Yeh character in a Habash-Soudi-Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_teh_marbuta_ar(s)

Normalize all occurences of Teh Marbuta characters to a Heh character in an Arabic string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_teh_marbuta_bw(s)

Normalize all occurences of Teh Marbuta characters to a Heh character in a Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_teh_marbuta_safebw(s)

Normalize all occurences of Teh Marbuta characters to a Heh character in a Safe Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_teh_marbuta_xmlbw(s)

Normalize all occurences of Teh Marbuta characters to a Heh character in a XML Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_teh_marbuta_hsb(s)

Normalize all occurences of Teh Marbuta characters to a Heh character in a Habash-Soudi-Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_ar(s)

Normalize various Alef variations to plain a Alef character in an Arabic string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_bw(s)

Normalize various Alef variations to plain a Alef character in a Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_safebw(s)

Normalize various Alef variations to plain a Alef character in a Safe Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_xmlbw(s)

Normalize various Alef variations to plain a Alef character in a XML Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str

camel_tools.utils.normalize.normalize_alef_hsb(s)

Normalize various Alef variations to plain a Alef character in a Habash-Soudi-Buckwalter encoded string.

Parameters:: s (str) – The string to be normalized.
Returns:: The normalized string.
Return type:: str