SliTaz Forum

llev

Member

Registered: 2011-12-09

Posts: 568

"read-mo.c: invalid multibyte sequence" when cooking retawq

Hi,

I've just turned on SSL support in retawq configure flags to repair a long-standing bug with our "man" (it browses linux.die.net, which has become available only as https).

But I found a bug that happens when cooking the pckage (bug already existed before my change). See the cooker log: after compilation,

[c]

Normalizing mo files...

read-mo.c:249: invalid multibyte sequence

(repeated several hundred times with same line number 249)

/tmp/tmp.nDlT5a.awk:89: missing 'msgstr' section

(repeated 20 times with varying line number)

msgfmt: too many errors, aborting

Error processing /home/slitaz/wok/retawq/install/usr/share/locale/ja/LC_MESSAGES/retawq.mo

read-mo.c:249: invalid multibyte sequence

(several hundred times again)

Done

  Time: 2.24s. Size: 135356 B -> 132947 B. Save: 2 KB.

================================================================================

[/c]

I don't know who outputs all these messages, so I can hardly debug this. I looked at file ja.po and didn't find a missing msgstr; maybe a quoting problem?

lexeii

Administrator

Registered: 2012-03-21

Posts: 1,853

Re: "read-mo.c: invalid multibyte sequence" when cooking retawq

Hi Lucas,

> I don't know who outputs all these messages

It's /usr/libexec/cookutils/compressor which tries to convert all the *.mo files to UTF-8 (it frees us from the need of having locale definition files).

I'm not sure, but looks like encodings (ISO-8859-1 and EUC-JP there) don't supported. Today I have no more time to investigate it, but tomorrow I'll try to find it out.

To disable conversion of *.mo files to UTF-8 please add this line to your receipt:

[c]COOKOPTS="!monorm"[/c]

Additional reading: http://cook.slitaz.org/next/cookutils/doc/cookopts.txt (search for "!monorm" on the page).

lexeii

Administrator

Registered: 2012-03-21

Posts: 1,853

Re: "read-mo.c: invalid multibyte sequence" when cooking retawq

Found the solution.

That pre-generated *.mo files shipped with retawq package are defective - they are missed header inside. Header contains very important info in the line (example for German catalog):

[c]"Content-Type: text/plain; charset=ISO-8859-1\n"[/c]
So, these *.mo files contains only [c]msgid[/c] and [c]msgstr[/c] without specifying which charset used, and in this case Gettext use ASCII charset, and any non-latin letter produce error. 17798 errors at all.

Solution is to re-generate *.mo files. I've put this code at the beginning of [c]compile_rules()[/c] and now all works like a charm, no more errors:

[c]8<----------------------------------------
BUILD_DEPENDS="ncurses-dev gettext-dev"

compile_rules() {
    # regenerate *.mo files (shipped ones are missed headers that lead to
    # thousand of errors when compressor normalize catalogs)
    for i in de es fr ja pt_BR; do
        rm i18n/$i.mo
        msgfmt -o i18n/$i.mo i18n/$i.po
    done

    ./configure \
. . . . . . . . . . . .
8<----------------------------------------[/c]
PS. I've found "invalid multibyte sequence" also in these packages logs in the SliTaz Cooking:

[*]gadmin-proftpd

[*]fpc

[*]python-formalchemy

[*]openerp-client

[*]boinc

And in addition in the SliTaz Next logs:

[*]python-formencode

[*]sweethome3d

Seems solution will be the similar...