?

Log in

No account? Create an account
Perl help needed - abates
Brilliant but slightly odd but very nice

Alden Bates
Date: 2010-02-15 20:09
Subject: Perl help needed
Security: Public
Tags:programming is a bastard
Here's my problem. I have a plugin I've written for Movable Type, which takes the page MT generates and splits it up into a bunch of subpages. The problem is that in doing this all the UTF8 characters get boned and converted into US-ASCII or something.

For instance, ß, which in UTF8 is 0xC39F, turns into 0xDF.

How do I get it to stop doing this? For the short term, I have inserted calls to utf8::encode from the UTF8 module before I write the string out, to convert them back to UTF8, however I'd really like to fix the thing so it doesn't convert the strings to US-ASCII in the first place.

From what I can tell, Perl does something to mark strings as UTF8 somehow, so I should maybe do something along those lines? I don't know.
Post A Comment | 6 Comments | Share | Link






Pete
User: theta_g
Date: 2010-02-15 08:04 (UTC)
Subject: (no subject)
How's this: invoke the utf8 pragma then your world is beautf8-ul

http://perldoc.perl.org/utf8.html
Reply | Thread | Link



Alden Bates
User: abates
Date: 2010-02-15 09:46 (UTC)
Subject: (no subject)
Adding "use utf8;" into the script doesn't seem to make a difference. I gather from reading the documentation that it's purely to tell perl that the script itself is in UTF8, not the variables it's manipulating.
Reply | Parent | Thread | Link



Michael
User: foeclan
Date: 2010-02-15 15:43 (UTC)
Subject: (no subject)
My sister (chesneyj) pointed me your way.

Your locale may be set to US-ASCII, thus defaulting to that when you write out the other pages. Check out this page for some more info. Try explicitly setting the locale to UTF8 and see if that fixes it.
Reply | Thread | Link



Alden Bates
User: abates
Date: 2010-02-16 04:35 (UTC)
Subject: (no subject)
Thanks for commenting. I had a try of that, specifically I added "setlocale(LC_CTYPE, "en_US.UTF-8");" to the point in my code before it does all the processing, but as far as I can tell, it didn't make any difference. :(
Reply | Parent | Thread | Link



Michael
User: foeclan
Date: 2010-02-16 05:33 (UTC)
Subject: (no subject)
Which OS and what version of Perl?

Did you go through the 'preparing to use locales' section on that page? There's some prepwork you need to do before the setlocale, and the locale stuff might not even be supported.
Reply | Parent | Thread | Link



Alden Bates
User: abates
Date: 2010-02-16 08:18 (UTC)
Subject: (no subject)
Perl 5.8.8 running on Linux 2.6

I'll have to check with my web host about most of that setup, as I don't have anything in the way of shell access, which is what's required for most of those checks.
Reply | Parent | Thread | Link



browse
Journal
links
August 2016