Re: [GENERAL] Invalid byte sequence for encoding UTF-8 0xc3\n

[Igor Korot]

John et al,

On Mon, Jul 31, 2017 at 12:13 AM, Igor Korot  wrote:
> John,
>
> On Sun, Jul 30, 2017 at 5:32 PM, Igor Korot  wrote:
>> Hi, John,
>>
>> On Sun, Jul 30, 2017 at 4:53 PM, John R Pierce  wrote:
>>> On 7/30/2017 1:43 PM, Igor Korot wrote:
>>>
>>> what encodings are default on your system ?`\l+` in psql should show the
>>> encodings.
>>>
>>> Is this "backslash + pipe + plus-sign"?
>>>
>>> Trying it gives: "Invalid command".
>>>
>>>
>>> \ + lower case L + plus sign, thats the psql metacommand to list all
>>> databases with extra info
>>>
>>> postgres=# \l+
>>>List
>>> of databases
>>>   Name  |   Owner   | Encoding |   Collate   |Ctype|
>>> Access privileges   |  Size   | Tablespace |Description
>>> +---+--+-+-+---+-++
>>>  junk   | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 6586 kB | pg_default |
>>>  observers_test | observers | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 207 MB  | pg_default |
>>>  pierce | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 6786 kB | pg_default |
>>>  postgres   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 6610 kB | pg_default | default administrative connection database
>>>  scac   | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 75 MB   | pg_default |
>>>  scac_save  | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 105 MB  | pg_default |
>>>  smf| smf   | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 34 MB   | pg_default |
>>>  sympa  | sympa | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 6898 kB | pg_default |
>>>  template0  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> =c/postgres  +| 6457 kB | pg_default | unmodifiable empty database
>>> |   |  | | |
>>> postgres=CTc/postgres | ||
>>>  template1  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> =c/postgres  +| 6465 kB | pg_default | default template for new
>>> databases
>>> |   |  | | |
>>> postgres=CTc/postgres | ||
>>>  tendenci   | tendenci  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 15 MB   | pg_default |
>>>  test   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>>> | 6634 kB | pg_default |
>>> (12 rows)
>>>
>>>
>>> for instance, all my databases are UTF8 on that server.
>>
>> Looks like mine are as well:
>>
>> draft=# \l+
>>List of 
>> databases
>>Name|  Owner   | Encoding | Collate | Ctype |   Access
>> privileges   |  Size   | Tablespace |Description
>> ---+--+--+-+---+---+-++
>>  draft | postgres | UTF8 | C   | C |
>> | 11 MB   | pg_default |
>>  leagues   | postgres | UTF8 | C   | C |
>> | 5865 kB | pg_default |
>>  postgres  | postgres | UTF8 | C   | C |
>> | 5865 kB | pg_default | default administrative connection
>> database
>>  template0 | postgres | UTF8 | C   | C | =c/postgres
>>+| 5865 kB | pg_default | unmodifiable empty database
>>|  |  | |   |
>> postgres=CTc/postgres | ||
>>  template1 | postgres | UTF8 | C   | C | =c/postgres
>>+| 5865 kB | pg_default | default template for new databases
>>|  |  | |   |
>> postgres=CTc/postgres | ||
>> (5 rows)
>>
>> I'm using the draft one.
>
> And my application is compiled in UNICODE mode.
>
> Now IIUC, in order to work correctly, the application and the database
> should have
> the same encoding. If I'm right, what is the way to get the encoding
> of the database?
>
> Thank you.

My application has a different ways to connect to the database.
One way is thru the ODBC another is thru the native client library.

When trying to connect using ODBC (iODBC + PostgreSQL driver)
everything works correctly. There is no errors.
However, when trying to connect with libpq, I get the aforementioned error.

I tried to call:

setlocale( LC_ALL, "" );

before and after I get the connection as suggested by ODBC driver maintainer,
Hiroshi Inoue, but it didn't help.

Is there anything else I can try to fix the problem?

Thank you.

>
>>
>> Thank you.
>>
>>>
>>>
>>> --
>>> john r pierce, recycling bits in santa cruz


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:

Re: [GENERAL] Invalid byte sequence for encoding UTF-8 0xc3\n

[Igor Korot]

John,

On Sun, Jul 30, 2017 at 5:32 PM, Igor Korot  wrote:
> Hi, John,
>
> On Sun, Jul 30, 2017 at 4:53 PM, John R Pierce  wrote:
>> On 7/30/2017 1:43 PM, Igor Korot wrote:
>>
>> what encodings are default on your system ?`\l+` in psql should show the
>> encodings.
>>
>> Is this "backslash + pipe + plus-sign"?
>>
>> Trying it gives: "Invalid command".
>>
>>
>> \ + lower case L + plus sign, thats the psql metacommand to list all
>> databases with extra info
>>
>> postgres=# \l+
>>List
>> of databases
>>   Name  |   Owner   | Encoding |   Collate   |Ctype|
>> Access privileges   |  Size   | Tablespace |Description
>> +---+--+-+-+---+-++
>>  junk   | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 6586 kB | pg_default |
>>  observers_test | observers | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 207 MB  | pg_default |
>>  pierce | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 6786 kB | pg_default |
>>  postgres   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 6610 kB | pg_default | default administrative connection database
>>  scac   | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 75 MB   | pg_default |
>>  scac_save  | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 105 MB  | pg_default |
>>  smf| smf   | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 34 MB   | pg_default |
>>  sympa  | sympa | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 6898 kB | pg_default |
>>  template0  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres  +| 6457 kB | pg_default | unmodifiable empty database
>> |   |  | | |
>> postgres=CTc/postgres | ||
>>  template1  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres  +| 6465 kB | pg_default | default template for new
>> databases
>> |   |  | | |
>> postgres=CTc/postgres | ||
>>  tendenci   | tendenci  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 15 MB   | pg_default |
>>  test   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
>> | 6634 kB | pg_default |
>> (12 rows)
>>
>>
>> for instance, all my databases are UTF8 on that server.
>
> Looks like mine are as well:
>
> draft=# \l+
>List of 
> databases
>Name|  Owner   | Encoding | Collate | Ctype |   Access
> privileges   |  Size   | Tablespace |Description
> ---+--+--+-+---+---+-++
>  draft | postgres | UTF8 | C   | C |
> | 11 MB   | pg_default |
>  leagues   | postgres | UTF8 | C   | C |
> | 5865 kB | pg_default |
>  postgres  | postgres | UTF8 | C   | C |
> | 5865 kB | pg_default | default administrative connection
> database
>  template0 | postgres | UTF8 | C   | C | =c/postgres
>+| 5865 kB | pg_default | unmodifiable empty database
>|  |  | |   |
> postgres=CTc/postgres | ||
>  template1 | postgres | UTF8 | C   | C | =c/postgres
>+| 5865 kB | pg_default | default template for new databases
>|  |  | |   |
> postgres=CTc/postgres | ||
> (5 rows)
>
> I'm using the draft one.

And my application is compiled in UNICODE mode.

Now IIUC, in order to work correctly, the application and the database
should have
the same encoding. If I'm right, what is the way to get the encoding
of the database?

Thank you.

>
> Thank you.
>
>>
>>
>> --
>> john r pierce, recycling bits in santa cruz


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Invalid byte sequence for encoding UTF-8 0xc3\n

[Igor Korot]

Hi, John,

On Sun, Jul 30, 2017 at 4:53 PM, John R Pierce  wrote:
> On 7/30/2017 1:43 PM, Igor Korot wrote:
>
> what encodings are default on your system ?`\l+` in psql should show the
> encodings.
>
> Is this "backslash + pipe + plus-sign"?
>
> Trying it gives: "Invalid command".
>
>
> \ + lower case L + plus sign, thats the psql metacommand to list all
> databases with extra info
>
> postgres=# \l+
>List
> of databases
>   Name  |   Owner   | Encoding |   Collate   |Ctype|
> Access privileges   |  Size   | Tablespace |Description
> +---+--+-+-+---+-++
>  junk   | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 6586 kB | pg_default |
>  observers_test | observers | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 207 MB  | pg_default |
>  pierce | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 6786 kB | pg_default |
>  postgres   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 6610 kB | pg_default | default administrative connection database
>  scac   | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 75 MB   | pg_default |
>  scac_save  | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 105 MB  | pg_default |
>  smf| smf   | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 34 MB   | pg_default |
>  sympa  | sympa | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 6898 kB | pg_default |
>  template0  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> =c/postgres  +| 6457 kB | pg_default | unmodifiable empty database
> |   |  | | |
> postgres=CTc/postgres | ||
>  template1  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> =c/postgres  +| 6465 kB | pg_default | default template for new
> databases
> |   |  | | |
> postgres=CTc/postgres | ||
>  tendenci   | tendenci  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 15 MB   | pg_default |
>  test   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> | 6634 kB | pg_default |
> (12 rows)
>
>
> for instance, all my databases are UTF8 on that server.

Looks like mine are as well:

draft=# \l+
   List of databases
   Name|  Owner   | Encoding | Collate | Ctype |   Access
privileges   |  Size   | Tablespace |Description
---+--+--+-+---+---+-++
 draft | postgres | UTF8 | C   | C |
| 11 MB   | pg_default |
 leagues   | postgres | UTF8 | C   | C |
| 5865 kB | pg_default |
 postgres  | postgres | UTF8 | C   | C |
| 5865 kB | pg_default | default administrative connection
database
 template0 | postgres | UTF8 | C   | C | =c/postgres
   +| 5865 kB | pg_default | unmodifiable empty database
   |  |  | |   |
postgres=CTc/postgres | ||
 template1 | postgres | UTF8 | C   | C | =c/postgres
   +| 5865 kB | pg_default | default template for new databases
   |  |  | |   |
postgres=CTc/postgres | ||
(5 rows)

I'm using the draft one.

Thank you.

>
>
> --
> john r pierce, recycling bits in santa cruz


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Invalid byte sequence for encoding UTF-8 0xc3\n

[John R Pierce]

On 7/30/2017 1:43 PM, Igor Korot wrote:

what encodings are default on your system ?`\l+` in psql should show the
encodings.

Is this "backslash + pipe + plus-sign"?

Trying it gives: "Invalid command".



\ + lower case L + plus sign, thats the psql metacommand to list all 
databases with extra info


postgres=# \l+
List of databases
  Name  |   Owner   | Encoding |   Collate   | Ctype|   
Access privileges   |  Size   | Tablespace |Description

+---+--+-+-+---+-++
 junk   | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 6586 kB | pg_default |
 observers_test | observers | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 207 MB  | pg_default |
 pierce | pierce| UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 6786 kB | pg_default |
 postgres   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 6610 kB | pg_default | default administrative 
connection database
 scac   | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 75 MB   | pg_default |
 scac_save  | scac  | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 105 MB  | pg_default |
 smf| smf   | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 34 MB   | pg_default |
 sympa  | sympa | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 6898 kB | pg_default |
 template0  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 | 
=c/postgres  +| 6457 kB | pg_default | unmodifiable empty database
|   |  | | | 
postgres=CTc/postgres | ||
 template1  | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 | 
=c/postgres  +| 6465 kB | pg_default | default template for new 
databases
|   |  | | | 
postgres=CTc/postgres | ||
 tendenci   | tendenci  | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 15 MB   | pg_default |
 test   | postgres  | UTF8 | en_US.UTF-8 | en_US.UTF-8 
|   | 6634 kB | pg_default |

(12 rows)


for instance, all my databases are UTF8 on that server.


--
john r pierce, recycling bits in santa cruz

Re: [GENERAL] Invalid byte sequence for encoding UTF-8 0xc3\n

[Igor Korot]

Hi, John,

On Sun, Jul 30, 2017 at 4:34 PM, John R Pierce  wrote:
> On 7/30/2017 1:19 PM, Igor Korot wrote:
>>
>> I am using a database for my project that I created inside SQLite3.
>> This database contains a table called "abc" (it is "abc" +
>> symbol with the code 225 -
>> greek letter "beta or a German symbol for "ss").
>
>
> in what encoding?   in ISO 8859-1, -15, beta aka sharp S is code 223
> (U+00DF), not 225.  in UTF-8, its C3,9F.
>
>> ...
>> Both the database and the table are created with default encoding.
>
>
> what encodings are default on your system ?`\l+` in psql should show the
> encodings.

Is this "backslash + pipe + plus-sign"?

Trying it gives: "Invalid command".

Thank you.

>
>
> --
> john r pierce, recycling bits in santa cruz
>
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general


-- 
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] Invalid byte sequence for encoding UTF-8 0xc3\n

[John R Pierce]

On 7/30/2017 1:19 PM, Igor Korot wrote:

I am using a database for my project that I created inside SQLite3.
This database contains a table called "abc" (it is "abc" +
symbol with the code 225 -
greek letter "beta or a German symbol for "ss").


in what encoding?   in ISO 8859-1, -15, beta aka sharp S is code 223 
(U+00DF), not 225.  in UTF-8, its C3,9F.



...
Both the database and the table are created with default encoding.


what encodings are default on your system ?`\l+` in psql should show 
the encodings.



--
john r pierce, recycling bits in santa cruz



--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general