From owner-FreeBSD-tech-jp@jp.freebsd.org  Wed Dec 27 21:54:44 2000
Received: (from daemon@localhost)
	by castle.jp.freebsd.org (8.9.3+3.2W/8.7.3) id VAA77491;
	Wed, 27 Dec 2000 21:54:44 +0900 (JST)
	(envelope-from owner-FreeBSD-tech-jp@jp.FreeBSD.org)
Received: from blizzard.sabbo.net (IDENT:root@ns.sabbo.net [193.193.218.18])
	by castle.jp.freebsd.org (8.9.3+3.2W/8.7.3) with ESMTP id VAA77484
	for <FreeBSD-tech-jp@jp.FreeBSD.org>; Wed, 27 Dec 2000 21:54:35 +0900 (JST)
	(envelope-from sobomax@FreeBSD.org)
Received: from vic.sabbo.net (root@vic.sabbo.net [193.193.218.109])
	by blizzard.sabbo.net (8.10.1/8.10.1) with ESMTP id eBRCs7Z19359;
	Wed, 27 Dec 2000 14:54:11 +0200
Received: from FreeBSD.org (big_brother.vega.com [192.168.1.1])
	by vic.sabbo.net (8.11.1/8.9.3) with ESMTP id eBRCs2D52692;
	Wed, 27 Dec 2000 14:54:03 +0200 (EET)
	(envelope-from sobomax@FreeBSD.org)
Message-ID: <3A49E668.FF0E701D@FreeBSD.org>
Date: Wed, 27 Dec 2000 14:54:00 +0200
From: Maxim Sobolev <sobomax@FreeBSD.org>
Organization: Vega International Capital
X-Mailer: Mozilla 4.76 [en] (WinNT; U)
X-Accept-Language: uk,ru,en
MIME-Version: 1.0
To: Motomichi Matsuzaki <mzaki@e-mail.ne.jp>
CC: current@FreeBSD.org, bp@FreeBSD.org, audit@FreeBSD.org,
        FreeBSD-tech-jp@jp.FreeBSD.org
References: <3A49BF05.9098A5D8@FreeBSD.org> <86wvcmm4kc.wl@tkc.att.ne.jp>
Content-Type: text/plain; charset=koi8-r
Content-Transfer-Encoding: 7bit
Reply-To: FreeBSD-tech-jp@jp.freebsd.org
Precedence: list
X-Distribute: distribute version 2.1 (Alpha) patchlevel 24e+000315
X-Sequence: FreeBSD-tech-jp 2991
Subject: [FreeBSD-tech-jp 2991] Re: Unicode support in cd9660 [patch for review]
Errors-To: owner-FreeBSD-tech-jp@jp.freebsd.org
Sender: owner-FreeBSD-tech-jp@jp.freebsd.org
X-Originator: sobomax@FreeBSD.org

Motomichi Matsuzaki wrote:

> At Wed, 27 Dec 2000 12:05:57 +0200,
> Maxim Sobolev <sobomax@FreeBSD.org> wrote:
> > Several days ago I got a CD with Russian filenames on it and discovered that
> > I'm unable to read those filenames. After some hacking I produced a patch,
>
> Vladimir Kushnir's patch will be for it.
>
> http://www.freebsd.org/cgi/getmsg.cgi?fetch=270425+0+/usr/local/www/db/text/2000/freebsd-hackers/20001203.freebsd-hackers
>
> and it is based on my patch:
>
> http://triaez.kaisei.org/~mzaki/joliet/
>
> > which should solve this problem in the manner similar to what we have in
> > msdosfs module (i.e. user-provided conversion table). I have to emphasize that
> > it's a temporary solution until we will have iconv support in kernel.
>
> *PLEASE* be careful about filename I18N.
>
> 1. Joliet extension
>
> The Joliet extension are built on Unicode basis,
> and is the "multilingual" filesystem.
> We can found CDs which contain files named by all of
> English, French, Russian, Chinese, and Japanese languages.
> So charset conversion per mount is not sufficient.

You can specify multiple charset conversion tables for each mount point, the problem is only to create appropriate conversion
tables (I do not have any CDs with anything than English/Russian filenames :-> ).

> 3. Relation to userland applications
>
> Currently, conversion table between Unicode and local charset are
> widely needed and implemented, for such as the Joliet extension,
> the FAT filesystem, TrueType rasterizers, WWW browsers, and so on.
> We should share the tables as possible for their consintency.
> So the ideal solution to code conversion are not in-kernel table
> but userland shared library.
> Therefore, filename code conversion should also be done in userland
> as possible.
>
> 4. Rough idea of me
>
> My preliminary idea to the filesystem I18N:
>
> * filenames recorded on Unix filesystems (e.g. FFS, MFS) use
>   an arbitrary codeset, for example Unicode.
>
> * interface between kernel and userland should use
>   filesystem-safe encoding, for example UTF-8.
>
> * userland applications can convert from/to the user-requested
>   charsets, such as latin-2, koi8, and euc-jp, using shared library.
>
> * the Joliet extension and UDF, which based on Unicode, need
>   no in-kernel conversion, in case Unix filesystems use Unicode.
>
> * the FAT filesystem, which use both Unicode and conventional
>   codepages, requires in-kernel conversion in order to
>   write the conventional 8.3 names.
>
> Any ideas?

Thanks for the pointing out, but I think that your proposal is too generic to be committed any time soon (not even to mention
MFC'ing it). Moreover, as I pointed out, currently efforts to provide generic Unicode functionality in kernel/userland are
underway, so it is likely that part of your work will be duplicated/obsoleted.

What I'm proposing here is quick'n'dirty (and limited as so) solution to allow mounting CD's with unicode filenames on it.
This solution is targeted to be temporary until iconv-based kernel interfaces will appear.

-Maxim

