문자셋(character set) 변환

1. character set 정보는 어디서??
character set 의 배정은 IANA (Internet Assigned Numbers Authority)에서 합니다.
최신의 character-set 정보는 다음 주소에서 찾아보시구요.
http://www.iana.org/assignments/character-sets

2. character set 변환은 왜?
다국어 지원을 하는 경우에는 이 character set을 고려해야 하는 경우가 있습니다.
중국 시스템에서 한국 시스템으로 어떤 문자를 보내고자 할때,
보통 중국은 big5 인코딩 방식을 사용하고, 한국은 ksc_5601을 사용합니다.
big5로 인코딩되어 있는 문자(한자가 되겠죠?)를 ksc_5601에 맞춰서 출력하며
이상한 값이 출력되겠죠?

3. 변환 방법은?
각 인코딩 방식마다 저마다의 인코딩 룰이 있습니다. utf-8 에서 unicode(ucs-2)로
변환하는 경우, 두 인코딩 방식을 비교하여 변화하는 코드를 작성해야겠죠.
일일이 인코딩 방식을 봐가면서 하기에는 다소 무리가 있는듯 합니다.
그래서, 편하게 사용할 수 있는 방법이 unix 시스템의 경우,
iconv 라는게 있습니다.

4. iconv 사용법은?
- 명령어
  사용법 : iconv -f 원시코드 -t 목적코드 [파일...]
  console에서 직접 iconv 명령어를 사용하여 변환할 수 있습니다.

- iconv library 이용
  프로그램을 작성하는 경우, 라이브러리를 사용하시면 됩니다.
  libiconv.a
  * 컴파일시 -liconv 주어야 합니다.
  * iconv_open, iconv, iconv_close 등의 함수가 있습니다.
    자세한 사항은 iconv man page를 참조하세요
 
5. iconv를 통해 변화할 수 있는 character set 정보는 어디에?
시스템마다 조금씩 다릅니다.
- compaq /usr/lib/nls/loc/iconv 화일 이름에서 _ 를 중심으로 왼쪽은 from, 오른쪽은 target - sun /usr/lib/iconv 화일 이름에서 % 를 중심으로 왼쪽은 from, 오른쪽은 target

* iconv man page
iconv(3) iconv(3) NAME iconv - Converts a string of characters from one codeset to another codeset LIBRARY The iconv library (libiconv) SYNOPSIS The following syntax is for pre-XSH5.0-compliant interfaces on Tru64 UNIX V4 and V5 systems: #include <iconv.h> size_t iconv( iconv_t cd, const char **inbuf, size_t *inbytesleft, char **outbuf, size_t *outbytesleft); The following syntax is for pre-V4 Tru64 UNIX systems and XSH5.0-compliant interfaces on V5 systems: #include <iconv.h> size_t iconv( iconv_t cd, char **inbuf, size_t *inbytesleft, char **outbuf, size_t *outbytesleft); STANDARDS Interfaces documented on this reference page conform to industry standards as follows: iconv(): XSH4.0, XSH4.2, XSH5.0 Refer to the standards(5) reference page for more information about indus- try standards and associated tags. PARAMETERS cd Specifies the conversion descriptor that points to the correct codeset converter inbuf Points to a variable that points to the beginning of a buffer that con- tains the characters to be converted inbytesleft Points to an integer that contains the number of bytes in inbuf still to be converted outbuf Points to a variable that points to the buffer that contains the characters that have been converted outbytesleft Points to an integer that contains the number of free bytes in outbuf DESCRIPTION The iconv() function converts a string of characters in inbuf into a dif- ferent codeset and returns the results in outbuf. The required converter is identified by cd, which must be a valid descriptor returned by a previous successful call to the iconv_open() function. On calling, the inbytesleft parameter indicates the number of bytes in inbuf to be converted and outbytesleft indicates the number of available bytes in outbuf. For codesets that include shift-state sequences, a call to iconv() in which inbuf is or points to a null pointer places the cd conversion descriptor into its initial shift state. In this case, as long as outbuf is not (or does not point to) a null pointer, the call places in outbuf the byte sequence that changes the output buffer to its initial shift state. If the output buffer is not large enough to hold the entire reset sequence, the call fails and sets errno to [E2BIG]. Any subsequent calls in which inbuf is not (or does not point to) a null pointer cause the conversion to take place from the current state of the conversion descriptor. See the RES- TRICTIONS section for information about support of shift-state encoding. If a sequence of input bytes does not form a character that is valid in the input codeset, conversion stops after the previous successfully converted character. If the input buffer ends with an incomplete character or shift sequence, conversion stops after the last byte sequence that was success- fully converted to a character. If the output is not large enough to hold the entire sequence of converted characters, conversion stops just prior to the input byte sequence that would cause the output buffer to overflow. On return from the call: + The inbuf value is updated to point to the byte following the last byte used successfully in conversion + The inbytesleft value is decremented to reflect the number of input bytes still not converted + The outbuf value is updated to point to the byte following the last output byte of successfully converted data + The outbytesleft value is decremented to reflect the number of bytes still available in the output buffer. + For codesets that include shift-state encoding, the conversion descriptor is updated to reflect the shift state in effect at the end of the last successfully converted byte sequence. It is possible for input data to include a character that is valid in the input codeset but for which an identical character does not exist in the output codeset. The output character for such cases is defined by the con- verter that iconv() applies when converting from one particular codeset to another. In other words, the output character in this case can vary from one codeset converter to another. RESTRICTIONS Currently, the operating system does not include locales whose codesets use shift-state encoding. Some sections of this reference page refer to iconv() behavior with respect to conversion of shift sequences. This information is included only for your convenience in developing portable applications that run on multiple platforms, some of which may supply locales whose codesets do use shift-state encoding. RETURN VALUES The iconv() function updates the variables pointed to by the call arguments to reflect the extent of the conversion and returns the number of non- identical conversions performed. If the function is successful and converts the entire input string, the value pointed to by inbytesleft will be 0 (zero). If an error occurs, the function returns (size_t)-1 and sets errno to indi- cate the condition. ERRORS If any of the following conditions occur, the iconv() function sets errno to the corresponding value: [E2BIG] The outbuf buffer is too small to contain all the converted charac- ters. The character that causes the overflow is not converted and inbytesleft indicates the bytes left to be converted, including the character that caused the overflow. The inbuf parameter points to the first byte of the characters left to convert. [EBADF] The cd parameter does not specify a valid converter descriptor. [EILSEQ] An input character does not belong to the input codeset. No conver- sion is attempted on the invalid character and inbytesleft indicates the bytes left to be converted, including the first byte of the invalid character. The inbuf parameter points to the first byte of the invalid character sequence. The values of outbuf and outbytesleft are updated according to the The values of outbuf and outbytesleft are updated according to the number of characters that were previously converted. [EINVAL] The last character or shift sequence in the inbuf parameter was not complete. The inbytesleft parameter indicates the number of input bytes still not converted. RELATED INFORMATION Functions: iconv_close(3), iconv_open(3) Commands: iconv(1), genxlt(1) Others: iconv_intro(5), standards(5)


* PHP 에서의 변환
기본적으로 서버에서 iconv 함수를 지원한다면 @iconv함수를 이용하시면 됩니다.(iconv함수는 PHP 기본 함수가 아님니다)
하지만 지원하지 않는다면 위에서 소개한 콘솔을 이용하시면됩니다.

@iconv함수를 이용한 방법

euc-kr -> UTF-8
$str = @iconv("UHC","UTF-8",$str);

UTF-8 -> euc-kr
$str = @iconv("UTF-8","UHC",$str);

big5 -> UTF-8
$str = @iconv("big5","UTF-8",$str);

콘솔 iconv를 이용한 방법

   function euckr2utf8($str) { 
     $str = ereg_replace("\n","\\n",$str); 
     $str = exec('echo \''.$str.'\' |iconv -c -f uhc -t utf-8'); 
     return ereg_replace("\\\\n","\n",$str); 
   } 

   function utf82euckr($str) { 
     $str = ereg_replace("\n","\\n",$str); 
     $str = exec('echo \''.$str.'\' |iconv -c -f utf-8 -t uhc'); 
     return ereg_replace("\\\\n","\n",$str); 
   } 

euc-kr -> UTF-8
$str = euckr2utf8($str);

UTF-8 -> euc-kr
$str = utf82euckr($str)

* UTF-8 인지 확인하는 함수

   function isutf8($str) { 
     $i=0; 
     $len = strlen($str); 

     for ($i=0;$i<$len;$i++) { 
        $sbit = ord(substr($str,$i,1)); 
        if ($sbit < 128) { 
        } else if($sbit > 191 && $sbit < 224) { 
           $i++; 
        } else if($sbit > 223 && $sbit < 240) { 
           $i+=2; 
        } else if($sbit > 239 && $sbit < 248) { 
           $i+=3; 
        } else { 
           return 0; 
        } 
     } 
     return 1; 
   } 

ps. 위함수는 KEBIL이 작성하신것입니다.

* 자신의 호스팅 계정이 함수방법, 콘솔방법을 지원하는지 알아보기위한 소스

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>변환 방법 확인</title>
</head>
<body>
1. iconv() 함수 존재여부 확인<br />
Result: <?
$test2 = function_exists(iconv);
if($test2 == 1) echo "있음";
else echo "없음";
?><br /><br />

2. Consol iconv test<br />
Result: <?
$test =  exec('echo "112" |iconv -f euc-kr -t utf-8');
if($test=="112") echo "사용가용";
else echo "사용불가";
?><br /><br />
<?
if($test == 112 && $test2 ==1) {
	echo"방법1과 방법2를 사용할 수 있습니다.<br />방법1을 사용하는 것이 가장 쉽습니다.<br />단, 변환불가능한 문자(UTF-8에는 존재하지만, euc-kr에는 존재하지 않는 문자, 기호 등)가 있을 경우 변환 결과가 나오지 않을 수 있습니다. 그럴 경우에는 방법 2를 사용해 주세요.";
}
else if($test != 112 && $test2 ==0) { 
	echo"두 방법 모두 사용할 수 없습니다. 방법 3을 사용하세요.";
}
else if($test != 112 && $test2 ==1) {
	echo"방법1을 사용할 수 있습니다.";
}
else if($test == 112 && $test2 ==0) {
	echo "방법2를 사용할 수 있습니다.";
}
?>
</body>
</html>