CNS 11643

The CNS 11643 character set (Chinese National Standard 11643), also officially known as the "Chinese Standard Interchange Code" (中文標準交換碼), is officially the standard character set of the Republic of China. (In practice, variants of Big5 are de facto standard.)

CNS 11643 is a superset of ASCII designed to conform to ISO 2022. It contains 16 planes, so the maximum possible number of encodable characters is 16×94×94 = 141376. Planes 12 to 15 (35344 code points) are specifically designated for user-defined characters.

Unlike CCCII, the encoding of variant characters in CNS 11643 is not related.

EUC-TW is a representation of CNS 11643 in Extended Unix Code (EUC) form.

CNS 11643
Language(s)Traditional Chinese
StandardCNS 11643
ClassificationISO 2022, DBCS, CJK encoding
Encoding formatsEUC-TW, ISO-2022-CN-EXT
Partial encodings: Big5-2003, ISO-2022-CN
Other related encoding(s)Big5, CCCII

Relationship to Big5

Levels 1 and 2 of the Big5 encoding correspond mostly to CNS 11643 planes 1 and 2, respectively, with occasional differences in order. They can be mapped using a list of ranges.[1]

The Big5-2003 variant of Big5 is defined as a partial encoding of CNS 11643.

References

  1. ^ Ken Lunde (1996-07-12). "4.3: CJK Character Set Compatibility Issues - Chinese (Taiwan)". CJK.INF Version 2.1.

External links

Big5

Big-5 or Big5 is a Chinese character encoding method used in Taiwan, Hong Kong, and Macau for traditional Chinese characters.

The People's Republic of China (PRC), which uses simplified Chinese characters, uses the GB 18030 character set instead.

Big5 gets its name from the consortium of five companies in Taiwan that developed it.

CJK Compatibility Forms

CJK Compatibility Forms is a Unicode block containing vertical glyph variants for east Asian compatibility.

CJK Compatibility Ideographs Supplement

CJK Compatibility Ideographs Supplement is a Unicode block containing Han characters used only for roundtrip compatibility mapping with planes 3, 4, 5, 6, 7, and 15 of CNS 11643-1992.

CJK Unified Ideographs

The Chinese, Japanese and Korean (CJK) scripts share a common background, collectively known as CJK characters. In the process called Han unification, the common (shared) characters were identified and named CJK Unified Ideographs. As of Unicode 12.0, Unicode defines a total of 87,887 CJK Unified Ideographs.The terms ideographs or ideograms may be misleading, since the Chinese script is not strictly a pictographic or ideographic system.

Historically, Vietnam used Chinese ideographs too, so sometimes the abbreviation "CJKV" is used. This system was replaced by the Latin-based Vietnamese alphabet in the 1920s.

CJK characters

In internationalization, CJK is a collective term for the Chinese, Japanese, and Korean languages, all of which include Chinese characters and derivatives (collectively, CJK characters) in their writing systems. Occasionally, Vietnamese is included, making the abbreviation CJKV, since Vietnamese historically used Chinese characters as well. Collectively, the CJKV characters often include hànzì in Chinese, kanji, kana in Japanese, hanja, hangul in Korean, and hán tự or chữ nôm in Vietnamese.

Charset detection

Character encoding detection, charset detection, or code page detection is the process of heuristically guessing the character encoding of a series of bytes that represent text. The technique is recognised to be unreliable and is only used when specific metadata, such as a HTTP Content-Type: header is either not available, or is assumed to be untrustworthy.

This algorithm usually involves statistical analysis of byte patterns, like frequency distribution of trigraphs of various languages encoded in each code page that will be detected; such statistical analysis can also be used to perform language detection. This process is not foolproof because it depends on statistical data.

In general, incorrect charset detection leads to mojibake.

One of the few cases where charset detection works reliably is detecting UTF-8. This is due to the large percentage of invalid byte sequences in UTF-8, so that text in any other encoding that uses bytes with the high bit set is extremely unlikely to pass a UTF-8 validity test. However, badly written charset detection routines do not run the reliable UTF-8 test first, and may decide that UTF-8 is some other encoding. For example, it was common that web sites in UTF-8 containing the name of the German city München were shown as München.

UTF-16 is fairly reliable to detect due to the high number of newlines (U+000A) and spaces (U+0020) that should be found when dividing the data into 16-bit words, and the fact that few encodings use 16-bit words. This process is not foolproof; for example, some versions of the Windows operating system would mis-detect the phrase "Bush hid the facts" (without a newline) in ASCII as Chinese UTF-16LE.

Charset detection is particularly unreliable in Europe, in an environment of mixed ISO-8859 encodings. These are closely related eight-bit encodings that share an overlap in their lower half with ASCII. There is no technical way to tell these encodings apart and recognising them relies on identifying language features, such as letter frequencies or spellings.

Due to the unreliability of heuristic detection, it is better to properly label datasets with the correct encoding. HTML documents served across the web by HTTP should have their encoding stated out-of-band using the Content-Type: header.

Content-Type: text/html;charset=UTF-8

An isolated HTML document, such as one being edited as a file on disk, may imply such a header by a meta tag within the file:

or with a new meta type in HTML5

If the document is Unicode, then some UTF encodings explicitly label the document with an embedded initial byte order mark (BOM).

Chinese Character Code for Information Interchange

The Chinese Character Code for Information Interchange (Chinese: 中文資訊交換碼) or CCCII is a character set developed specifically to address the problem of interchange of Chinese information. It is used mostly by libraries because the code contains various properties considered to be desirable by libraries. Its 94 ISO 2022 planes are grouped into 16 layers of 6 planes each. Layer 2 contains Simplified Chinese characters, with their row and cell numbers being the same as the Traditional Chinese equivalent in layer 1. Layers 3 through 12 contain further variant forms, at row and cell numbers homologous to the first two layers. Layer 13 contains kana and Japanese kokuji, and layer 14 contains hangul.CCCII is a superset of ASCII designed to conform to ISO 2022. Each Chinese character is represented by a 3-byte code in which each byte is 7-bit. Thus, the maximum number of Chinese characters representable in CCCII is 94×94×94 = 830584. In practice the number of characters encodable by CCCII would be less than this number, because variant characters are encoded in related ISO 2022 planes under CCCII, so most of the code points would have to be reserved for variants.

The code is adopted as the character set for Library of Congress with the name East Asia Coded Character (EACC, ANSI/NISO Z39.64). However, some CCCII characters don't have EACC equivalent.

Code page 1287

Code page 1287, also known as CP1287, DEC Greek (8-bit) and EL8DEC, is one of the code pages implemented for the VT220 terminals. It supports the Greek language.

Code page 1288

Code page 1288, also known as CP1288, DEC Turkish (8-bit) and TR8DEC, is one of the code pages implemented for the VT220 terminals. It supports the Turkish language.

Extended Unix Code

Extended Unix Code (EUC) is a multibyte character encoding system used primarily for Japanese, Korean, and simplified Chinese.

The structure of EUC is based on the ISO-2022 standard, which specifies a way to represent character sets containing a maximum of 94 characters, or 8836 (942) characters, or 830584 (943) characters, as sequences of 7-bit codes. Only ISO-2022 compliant character sets can have EUC forms. Up to four coded character sets (referred to as G0, G1, G2, and G3 or as code sets 0, 1, 2, and 3) can be represented with the EUC scheme.

G0 is almost always an ISO-646 compliant coded character set such as US-ASCII, ISO 646:KR (KS X 1003) or ISO 646:JP (the lower half of JIS X 0201) that is invoked on GL (i.e. with the most significant bit cleared). An exception from US-ASCII is that 0x5C (backslash in US-ASCII) is often used to represent a Yen sign in EUC-JP (see below) and a Won sign in EUC-KR.

To get the EUC form of an ISO-2022 character, the most significant bit of each 7-bit byte of the original ISO 2022 codes is set (by adding 128 to each of these original 7-bit codes); this allows software to easily distinguish whether a particular byte in a character string belongs to the ISO-646 code or the ISO-2022 (EUC) code.

The most commonly used EUC codes are variable-width encodings with a character belonging to G0 (ISO-646 compliant coded character set) taking one byte and a character belonging to G1 (taken by a 94x94 coded character set) represented in two bytes. The EUC-CN form of GB2312 and EUC-KR are examples of such two-byte EUC codes. EUC-JP includes characters represented by up to three bytes whereas a single character in EUC-TW can take up to four bytes.

Modern applications are more likely to use UTF-8, which supports all of the glyphs of the EUC codes, and more, and is generally more portable with fewer vendor deviations and errors.

ISO/IEC 2022

ISO/IEC 2022 Information technology—Character code structure and extension techniques, is an ISO standard (equivalent to the ECMA standard ECMA-35) specifying

a technique for including multiple character sets in a single character encoding system, and

a technique for representing these character sets in both 7 and 8 bit systems using the same encoding.Many of the character sets included as ISO/IEC 2022 encodings are 'double byte' encodings where two bytes correspond to a single character. This makes ISO-2022 a variable width encoding. But a specific implementation does not have to implement all of the standard; the conformance level and the supported character sets are defined by the implementation.

ISO/IEC 6937

ISO/IEC 6937:2001, Information technology — Coded graphic character set for text communication — Latin alphabet, is a multibyte extension of ASCII, or rather of ISO/IEC 646-IRV. It was developed in common with ITU-T (then CCITT) for telematic services under the name of T.51, and first became an ISO standard in 1983. Certain byte codes are used as lead bytes for letters with diacritics (accents). The value of the lead byte often indicates which diacritic that the letter has, and the follow byte then has the ASCII-value for the letter that the diacritic is on. Only certain combinations of lead byte and follow byte are allowed, and there are some exceptions to the lead byte interpretation for some follow bytes. However, there are no combining characters at all are encoded in ISO/IEC 6937. But one can represent some free-standing diacritics, often by letting the follow byte have the code for ASCII space.

ISO/IEC 6937's architects were Hugh McGregor Ross, Peter Fenwick, Bernard Marti and Loek Zeckendorf.

ISO6937/2 defines 327 characters found in modern European languages using the Latin alphabet. Non-Latin European characters, such as Cyrillic and Greek, are not included in the standard. Also, some diacritics used with the Latin alphabet like the Romanian comma are not included, using cedilla instead as no distinction between cedilla and comma below was made at the time.

IANA has registered the charset names ISO_6937-2-25 and ISO_6937-2-add for two (older) versions of this standard (plus control codes). But in practice this character encoding is unused on the Internet.

The ISO/IEC 2022 escape sequence to specify the right-hand side of the ISO/IEC 6937 character set is ESC - R (hex 1B 2D 52).

ISO/IEC 8859-12

ISO/IEC 8859-12 would have been part 12 of the ISO/IEC 8859 character encoding standard series.

ISO 8859-12 was originally proposed to support the Celtic languages. ISO 8859-12 was later slated for Latin/Devanagari, but this was abandoned in 1997, during the 12th meeting of ISO/IEC JTC 1/SC 2/WG 3 in Iraklion-Crete, Greece, 4 to 7 July 1997. The Celtic proposal was changed to ISO 8859-14.

ISO/IEC 8859-16

ISO/IEC 8859-16:2001, Information technology — 8-bit single-byte coded graphic character sets — Part 16: Latin alphabet No. 10, is part of the ISO/IEC 8859 series of ASCII-based standard character encodings, first edition published in 2001. It is informally referred to as Latin-10 or South-Eastern European. It was designed to cover Albanian, Croatian, Hungarian, Polish, Romanian, Serbian and Slovenian, but also French, German, Italian and Irish Gaelic (new orthography).

ISO-8859-16 is the IANA preferred charset name for this standard when supplemented with the C0 and C1 control codes from ISO/IEC 6429.

Microsoft has assigned code page 28606 a.k.a. Windows-28606 to ISO-8859-16.

ISO/IEC 8859-3

ISO/IEC 8859-3:1999, Information technology — 8-bit single-byte coded graphic character sets — Part 3: Latin alphabet No. 3, is part of the ISO/IEC 8859 series of ASCII-based standard character encodings, first edition published in 1988. It is informally referred to as Latin-3 or South European. It was designed to cover Turkish, Maltese and Esperanto, though the introduction of ISO/IEC 8859-9 superseded it for Turkish. The encoding remains popular with users of Esperanto, though use is waning as application support for Unicode becomes more common.

ISO-8859-3 is the IANA preferred charset name for this standard when supplemented with the C0 and C1 control codes from ISO/IEC 6429. Microsoft has assigned code page 28593 a.k.a. Windows-28593 to ISO-8859-3 in Windows. IBM has assigned code page 913 to ISO 8859-3.

ISO/IEC 8859-9

ISO/IEC 8859-9:1999, Information technology — 8-bit single-byte coded graphic character sets — Part 9: Latin alphabet No. 5, is part of the ISO/IEC 8859 series of ASCII-based standard character encodings, first edition published in 1989. It is informally referred to as Latin-5 or Turkish. It was designed to cover the Turkish language, designed as being of more use than the ISO/IEC 8859-3 encoding. It is identical to ISO/IEC 8859-1 except for these six replacements of Icelandic characters with characters unique to the Turkish alphabet:

ISO-8859-9 is the IANA preferred charset name for this standard when supplemented with the C0 and C1 control codes from ISO/IEC 6429. In modern applications Unicode and UTF-8 are preferred. 0.1% of all web pages use ISO-8859-9 in February 2016.Microsoft has assigned code page 28599 a.k.a. Windows-28599 to ISO-8859-9 in Windows. IBM has assigned Code page 920 to ISO-8859-9.

Semicolon

The semicolon or semi colon (;) is a punctuation mark that separates major sentence elements. A semicolon can be used between two closely related independent clauses, provided they are not already joined by a coordinating conjunction. Semicolons can also be used in place of commas to separate items in a list, particularly when the elements of that list contain commas.

Small Form Variants (Unicode block)

Small Form Variants is a Unicode block containing small punctuation characters for compatibility with the Chinese National Standard CNS 11643.

Early telecommunications
ISO/IEC 8859
Bibliographic use
National standards
EUC
ISO/IEC 2022
MacOS code pages("scripts")
DOS code pages
IBM AIX code pages
IBM Apple MacIntoshemulations
IBM Adobe emulations
IBM DEC emulations
IBM HP emulations
Windows code pages
EBCDIC code pages
Platform specific
Unicode / ISO/IEC 10646
TeX typesetting system
Miscellaneous code pages
Related topics

This page is based on a Wikipedia article written by authors (here).
Text is available under the CC BY-SA 3.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.