subject:"\[Koha\-bugs\] \[Bug 18336\] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections"

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #74 from Jonathan Druart  
---
QA test updated:
commit 19e3654de795071116bfd064a467ced16e7d9c34
  Bug 18336: Check the charset collate for kohastructure.sql

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Jonathan Druart  changed:

   What|Removed |Added

 Status|Passed QA   |Pushed to Master

--- Comment #73 from Jonathan Druart  
---
Pushed to master for 18.05, thanks to everybody involved!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #72 from Jonathan Druart  
---
The only thing I found in the mysql doc is:

"""
 Tip: To save space with utf8mb4, use VARCHAR instead of CHAR. Otherwise, MySQL
must reserve four bytes for each character in a CHAR CHARACTER SET utf8mb4
column because that is the maximum possible length. For example, MySQL must
reserve 40 bytes for a CHAR(10) CHARACTER SET utf8mb4 column. 
"""

https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #71 from Victor Grousset/tuxayo  ---
(In reply to Victor Grousset/tuxayo from comment #69)
> So, isn't there a loss of data on VARCHAR ? I didn't see VARCHAR being
> converted to something else.

Nothing to worry about it seems.

https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-conversion.html

> Because utf8 cannot store the character at all, utf8 columns have no 
> supplementary characters and you need not worry about converting characters 
> or losing data when converting to utf8mb4.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #70 from Victor Grousset/tuxayo  ---
(In reply to Tomás Cohen Arazi from comment #68)
> (In reply to Victor Grousset/tuxayo from comment #67)
> > Can there be a loss of performance after "text" → "mediumtext"?
> > 
> > https://stackoverflow.com/questions/3516290/any-pitfalls-of-converting-mysql-
> > text-field-to-mediumtext
> > > with PHP 5.6 and PDO, mediumtext uses more memory than text (with same 
> > > value, an empty string). In my simple test, the difference is 0.5 MiB vs. 
> > > 1.5 MiB
> > 
> > Our ORM might have such an issue.
> 
> Can u think of a way to meassure that? I'm asking at #dbix-class@OFTC with
> no answers yet.

Is there anything that could keep alive for few seconds at least instances of
ORM objects? Maybe this isn't an issue then.

Well if some operations cause memory spikes of let's say 100MiB instead of
30MiB then that could be an issue.

Actually it could be worse:
https://stackoverflow.com/questions/3516290/any-pitfalls-of-converting-mysql-text-field-to-mediumtext

> Some client interface libraries pre-allocate a buffer to hold results, and 
> they allocate enough memory for the largest possible value, since the client 
> doesn't know the data until you fetch.

> Therefore the library would allocate 16MB per mediumtext while it would 
> allocate 64KB for a text. This is something to watch out for if you have a 
> low memory limit in your client layer. For instance, PHP has a memory_limit 
> config parameter for scripts, and the buffer allocated for data result sets 
> would count toward this.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #69 from Victor Grousset/tuxayo  ---

(In reply to M. Tompsett from comment #10)
> A lot of fields are shrunk from 255 to 191. This is because
> index size for VARCHAR on InnoDB has a max size, in bytes, of 767 bytes (or
> similar) and utf8 uses 3 bytes for each char, and utf8mb4, 4.
> 767/3 > 255. but 767/4 > 191.
> https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-conversion.html
> 
> Gain functionality (necessary for true unicode), lose space (not an issue
> yet).



So, isn't there a loss of data on VARCHAR ? I didn't see VARCHAR being
converted to something else.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #68 from Tomás Cohen Arazi  ---
(In reply to Victor Grousset/tuxayo from comment #67)
> Can there be a loss of performance after "text" → "mediumtext"?
> 
> https://stackoverflow.com/questions/3516290/any-pitfalls-of-converting-mysql-
> text-field-to-mediumtext
> > with PHP 5.6 and PDO, mediumtext uses more memory than text (with same 
> > value, an empty string). In my simple test, the difference is 0.5 MiB vs. 
> > 1.5 MiB
> 
> Our ORM might have such an issue.

Can u think of a way to meassure that? I'm asking at #dbix-class@OFTC with no
answers yet.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Victor Grousset/tuxayo  changed:

   What|Removed |Added

 CC||victor.grous...@biblibre.co
   ||m

--- Comment #67 from Victor Grousset/tuxayo  ---
Can there be a loss of performance after "text" → "mediumtext"?

https://stackoverflow.com/questions/3516290/any-pitfalls-of-converting-mysql-text-field-to-mediumtext
> with PHP 5.6 and PDO, mediumtext uses more memory than text (with same value, 
> an empty string). In my simple test, the difference is 0.5 MiB vs. 1.5 MiB

Our ORM might have such an issue.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #63 from Kyle M Hall  ---
Created attachment 71848
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71848=edit
Bug 18336: Add explicit index names to kohastructure.sql

This patch just adds an explicit index name to the search_marc_map
table. The atomicupdate on this patchset is a good example of why we
better have them.

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #65 from Kyle M Hall  ---
Created attachment 71850
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71850=edit
Bug 18336: Shift *TEXT columns size

Because of the 3-byte vs. 4-byte char size change in utf8mb4, altering a
column's encoding from utf8 into utf8mb4 results in this changes:

TEXT => MEDIUMTEXT
MEDIUMTEXT => LONGTEXT

The column size in the rows (the text itself goes to a separate object
storage) shifts by 1 byte, because all chars are shifted in bytes size
too, so there needs to be room for the byte-count in the column.

This is a debatable change, but the path needs to be included along with
the rest of the patchset for consistency.

Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #66 from Kyle M Hall  ---
Created attachment 71851
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71851=edit
Bug 18336: Library groups fix

Library groups were added after this patchset was submitted.
This patch adjusts kohastructure.sql entry for library_groups.

Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #64 from Kyle M Hall  ---
Created attachment 71849
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71849=edit
Bug 18336: Fix missing utf8_bin > utf8mb4_bin translation in kohastructure.sql

Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Katrin Fischer 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #62 from Kyle M Hall  ---
Created attachment 71847
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71847=edit
Bug 18336: Atomic update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #61 from Kyle M Hall  ---
Created attachment 71846
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71846=edit
Bug 18336: DBIC update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Kyle M Hall  changed:

   What|Removed |Added

  Attachment #71779|0   |1
is obsolete||
  Attachment #71780|0   |1
is obsolete||
  Attachment #71781|0   |1
is obsolete||
  Attachment #71782|0   |1
is obsolete||
  Attachment #71783|0   |1
is obsolete||
  Attachment #71784|0   |1
is obsolete||
  Attachment #71785|0   |1
is obsolete||
  Attachment #71786|0   |1
is obsolete||
  Attachment #71836|0   |1
is obsolete||

--- Comment #58 from Kyle M Hall  ---
Created attachment 71843
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71843=edit
Bug 18336: Full stack tests for supplemental UTF-8 chars

This patch introduces tests for Koha's support for 4-byte supplemental
UTF-8 chars. encoding/decoding tools handle this gracefuly. The missing
piece is the MySQL DB backend.

The tests in this patch:
- Adds a couple records for each flavour (MARC21 and UNIMARC) so
  search_utf8.t tests 4-byte chars are handled correctly
- Adds emoji testing in auth_values_input_www.t

To test:
- Apply this patch
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: It should fail if the DB hasn't been migrated into using
utf8mb4

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 
Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #59 from Kyle M Hall  ---
Created attachment 71844
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71844=edit
Bug 18336: SET NAMES utf8mb4 in Koha::Database

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 
Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #60 from Kyle M Hall  ---
Created attachment 71845
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71845=edit
Bug 18336: Convert schema from utf8 to utf8mb4

This patch adapts the DB structure so it uses utf8mb4 encoding and
utf8mb4_generic_ci collation.

Indexes for columns of type VARCHAR with prefix lenght higher than 191
are shortened because of the smaller max index lenght for utf8mb4

Note: please beware that testing this patchset risks your data and the
patchset includes reinitializing the DB.

To test:
- Be on the master branch
- Have a clean DB:
  $ reset_all
  (y)
- Apply the first patch (Unit tests)
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: Tests fail because Koha doesn't support supplemental (UTF-8)
 chars.
- Apply the rest of this patchset
- Upgrade the schema:
  $ kshell
 k$ perl installer/data/mysql/updatedatabase.pl
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!
- Now start from a clean DB
- Run:
  $ reset_all
  (y)
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!

Verify you can use emojis all over the place (MARC records, AV
descriptions, etc).

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 
Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Kyle M Hall 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Kyle M Hall  changed:

   What|Removed |Added

 Status|Signed Off  |Passed QA

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #57 from Tomás Cohen Arazi  ---
Created attachment 71836
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71836=edit
Bug 18336: Library groups fix

Library groups were added after this patchset was submitted.
This patch adjusts kohastructure.sql entry for library_groups.

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71214|0   |1
is obsolete||

--- Comment #53 from Tomás Cohen Arazi  ---
Created attachment 71783
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71783=edit
Bug 18336: Atomic update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71213|0   |1
is obsolete||

--- Comment #52 from Tomás Cohen Arazi  ---
Created attachment 71782
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71782=edit
Bug 18336: DBIC update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71211|0   |1
is obsolete||

--- Comment #50 from Tomás Cohen Arazi  ---
Created attachment 71780
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71780=edit
Bug 18336: SET NAMES utf8mb4 in Koha::Database

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 
Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71215|0   |1
is obsolete||

--- Comment #54 from Tomás Cohen Arazi  ---
Created attachment 71784
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71784=edit
Bug 18336: Add explicit index names to kohastructure.sql

This patch just adds an explicit index name to the search_marc_map
table. The atomicupdate on this patchset is a good example of why we
better have them.

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71216|0   |1
is obsolete||

--- Comment #55 from Tomás Cohen Arazi  ---
Created attachment 71785
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71785=edit
Bug 18336: Fix missing utf8_bin > utf8mb4_bin translation in kohastructure.sql

Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71337|0   |1
is obsolete||

--- Comment #56 from Tomás Cohen Arazi  ---
Created attachment 71786
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71786=edit
Bug 18336: Shift *TEXT columns size

Because of the 3-byte vs. 4-byte char size change in utf8mb4, altering a
column's encoding from utf8 into utf8mb4 results in this changes:

TEXT => MEDIUMTEXT
MEDIUMTEXT => LONGTEXT

The column size in the rows (the text itself goes to a separate object
storage) shifts by 1 byte, because all chars are shifted in bytes size
too, so there needs to be room for the byte-count in the column.

This is a debatable change, but the path needs to be included along with
the rest of the patchset for consistency.

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71212|0   |1
is obsolete||

--- Comment #51 from Tomás Cohen Arazi  ---
Created attachment 71781
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71781=edit
Bug 18336: Convert schema from utf8 to utf8mb4

This patch adapts the DB structure so it uses utf8mb4 encoding and
utf8mb4_generic_ci collation.

Indexes for columns of type VARCHAR with prefix lenght higher than 191
are shortened because of the smaller max index lenght for utf8mb4

Note: please beware that testing this patchset risks your data and the
patchset includes reinitializing the DB.

To test:
- Be on the master branch
- Have a clean DB:
  $ reset_all
  (y)
- Apply the first patch (Unit tests)
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: Tests fail because Koha doesn't support supplemental (UTF-8)
 chars.
- Apply the rest of this patchset
- Upgrade the schema:
  $ kshell
 k$ perl installer/data/mysql/updatedatabase.pl
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!
- Now start from a clean DB
- Run:
  $ reset_all
  (y)
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!

Verify you can use emojis all over the place (MARC records, AV
descriptions, etc).

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 
Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-16 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

  Attachment #71210|0   |1
is obsolete||

--- Comment #49 from Tomás Cohen Arazi  ---
Created attachment 71779
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71779=edit
Bug 18336: Full stack tests for supplemental UTF-8 chars

This patch introduces tests for Koha's support for 4-byte supplemental
UTF-8 chars. encoding/decoding tools handle this gracefuly. The missing
piece is the MySQL DB backend.

The tests in this patch:
- Adds a couple records for each flavour (MARC21 and UNIMARC) so
  search_utf8.t tests 4-byte chars are handled correctly
- Adds emoji testing in auth_values_input_www.t

To test:
- Apply this patch
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: It should fail if the DB hasn't been migrated into using
utf8mb4

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 
Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-08 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

 Status|Failed QA   |Signed Off

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-08 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #48 from Tomás Cohen Arazi  ---
Created attachment 71337
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71337=edit
Bug 18336: Shift *TEXT columns size

Because of the 3-byte vs. 4-byte char size change in utf8mb4, altering a
column's encoding from utf8 into utf8mb4 results in this changes:

TEXT => MEDIUMTEXT
MEDIUMTEXT => LONGTEXT

The column size in the rows (the text itself goes to a separate object
storage) shifts by 1 byte, because all chars are shifted in bytes size
too, so there needs to be room for the byte-count in the column.

This is a debatable change, but the path needs to be included along with
the rest of the patchset for consistency.

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-06 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #47 from Tomás Cohen Arazi  ---
(In reply to Jonathan Druart from comment #44)
> (In reply to Josef Moravec from comment #43)
> > (In reply to Jonathan Druart from comment #41)
> > > (In reply to Jonathan Druart from comment #40)
> > > > Confirmed, it fails for me as well.
> > > 
> > > And all green on master.
> > 
> > Not for me - without data, it fails on master the same way,
> > 
> > when I do reset_all on kohadevbox then it is green
> 
> I am always using misc4dev to populate the DB, so with the sample data

The sample data doesn't include biblios. The one containing them is misc4dev.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #46 from Tomás Cohen Arazi  ---
(In reply to Jonathan Druart from comment #45)
> This is certainly not expected:
> 
> Fresh install (with and without the patches):
> | biblio | CREATE TABLE `biblio` (
>   `biblionumber` int(11) NOT NULL AUTO_INCREMENT,
>   `frameworkcode` varchar(4) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
>   `author` mediumtext COLLATE utf8_unicode_ci,
>   `title` mediumtext COLLATE utf8_unicode_ci,
>   `unititle` mediumtext COLLATE utf8_unicode_ci,
>   `notes` mediumtext COLLATE utf8_unicode_ci,
> 
> Fresh master intall + upgrade:
> | biblio | CREATE TABLE `biblio` (
>   `biblionumber` int(11) NOT NULL AUTO_INCREMENT,
>   `frameworkcode` varchar(4) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
>   `author` longtext COLLATE utf8mb4_unicode_ci,
>   `title` longtext COLLATE utf8mb4_unicode_ci,
>   `unititle` longtext COLLATE utf8mb4_unicode_ci,
>   `notes` longtext COLLATE utf8mb4_unicode_ci,

It is interesting that the atomicupdate DOESN'T touch the biblio table besides
changing the collation and encoding!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #44 from Jonathan Druart  
---
(In reply to Josef Moravec from comment #43)
> (In reply to Jonathan Druart from comment #41)
> > (In reply to Jonathan Druart from comment #40)
> > > Confirmed, it fails for me as well.
> > 
> > And all green on master.
> 
> Not for me - without data, it fails on master the same way,
> 
> when I do reset_all on kohadevbox then it is green

I am always using misc4dev to populate the DB, so with the sample data

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #45 from Jonathan Druart  
---
This is certainly not expected:

Fresh install (with and without the patches):
| biblio | CREATE TABLE `biblio` (
  `biblionumber` int(11) NOT NULL AUTO_INCREMENT,
  `frameworkcode` varchar(4) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
  `author` mediumtext COLLATE utf8_unicode_ci,
  `title` mediumtext COLLATE utf8_unicode_ci,
  `unititle` mediumtext COLLATE utf8_unicode_ci,
  `notes` mediumtext COLLATE utf8_unicode_ci,

Fresh master intall + upgrade:
| biblio | CREATE TABLE `biblio` (
  `biblionumber` int(11) NOT NULL AUTO_INCREMENT,
  `frameworkcode` varchar(4) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
  `author` longtext COLLATE utf8mb4_unicode_ci,
  `title` longtext COLLATE utf8mb4_unicode_ci,
  `unititle` longtext COLLATE utf8mb4_unicode_ci,
  `notes` longtext COLLATE utf8mb4_unicode_ci,

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #43 from Josef Moravec  ---
(In reply to Jonathan Druart from comment #41)
> (In reply to Jonathan Druart from comment #40)
> > Confirmed, it fails for me as well.
> 
> And all green on master.

Not for me - without data, it fails on master the same way,

when I do reset_all on kohadevbox then it is green

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #42 from Jonathan Druart  
---
All green after an upgrade también!
So it only fails for new installs

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #41 from Jonathan Druart  
---
(In reply to Jonathan Druart from comment #40)
> Confirmed, it fails for me as well.

And all green on master.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Jonathan Druart  changed:

   What|Removed |Added

 Status|Signed Off  |Failed QA

--- Comment #40 from Jonathan Druart  
---
Confirmed, it fails for me as well.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #39 from Tomás Cohen Arazi  ---
(In reply to Josef Moravec from comment #38)
> (In reply to Katrin Fischer from comment #37)
> > It worked with a reset_all db, but didn't work when I tried a fresh install,
> > patches applied after my initial testing. (drop database, create database,
> > web installer with mandatory data + onboarding tool).
> > Didn't work = tests didn't pass.
> > That got me confused a bit so asking for a second opinion now.
> 
> I tried it too, recreate db manually. Added all mandatory and optional data,
> go through onboarding tool. And the test started failing.

It seems to me that then there's a problem with the tests, depending on
non-existent data. Can you please test if this is a problem with this patchset
or just master fails too?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #38 from Josef Moravec  ---
(In reply to Katrin Fischer from comment #37)
> It worked with a reset_all db, but didn't work when I tried a fresh install,
> patches applied after my initial testing. (drop database, create database,
> web installer with mandatory data + onboarding tool).
> Didn't work = tests didn't pass.
> That got me confused a bit so asking for a second opinion now.

I tried it too, recreate db manually. Added all mandatory and optional data, go
through onboarding tool. And the test started failing.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #37 from Katrin Fischer  ---
It worked with a reset_all db, but didn't work when I tried a fresh install,
patches applied after my initial testing. (drop database, create database, web
installer with mandatory data + onboarding tool).
Didn't work = tests didn't pass.
That got me confused a bit so asking for a second opinion now.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #36 from Tomás Cohen Arazi  ---
(In reply to Katrin Fischer from comment #25)
> Hm it worked on a clean db (of course) :)

If by clean you mean the result of reset_all, then the failure might be related
to not restarting the plack process, then the DB connection isn't properly
configured.

It is important to be sure the upgrade process goes smooth!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

 Status|Passed QA   |Signed Off

--- Comment #35 from Katrin Fischer  ---
Thought a bit more about it - could we get a second QA opinion please?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

 Status|Signed Off  |Passed QA

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #71022|0   |1
is obsolete||

--- Comment #34 from Katrin Fischer  ---
Created attachment 71216
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71216=edit
Bug 18336: Fix missing utf8_bin > utf8mb4_bin translation in kohastructure.sql

Signed-off-by: Tomas Cohen Arazi 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #70909|0   |1
is obsolete||

--- Comment #33 from Katrin Fischer  ---
Created attachment 71215
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71215=edit
Bug 18336: Add explicit index names to kohastructure.sql

This patch just adds an explicit index name to the search_marc_map
table. The atomicupdate on this patchset is a good example of why we
better have them.

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #70907|0   |1
is obsolete||

--- Comment #31 from Katrin Fischer  ---
Created attachment 71213
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71213=edit
Bug 18336: DBIC update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #70908|0   |1
is obsolete||

--- Comment #32 from Katrin Fischer  ---
Created attachment 71214
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71214=edit
Bug 18336: Atomic update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #70906|0   |1
is obsolete||

--- Comment #30 from Katrin Fischer  ---
Created attachment 71212
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71212=edit
Bug 18336: Convert schema from utf8 to utf8mb4

This patch adapts the DB structure so it uses utf8mb4 encoding and
utf8mb4_generic_ci collation.

Indexes for columns of type VARCHAR with prefix lenght higher than 191
are shortened because of the smaller max index lenght for utf8mb4

Note: please beware that testing this patchset risks your data and the
patchset includes reinitializing the DB.

To test:
- Be on the master branch
- Have a clean DB:
  $ reset_all
  (y)
- Apply the first patch (Unit tests)
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: Tests fail because Koha doesn't support supplemental (UTF-8)
 chars.
- Apply the rest of this patchset
- Upgrade the schema:
  $ kshell
 k$ perl installer/data/mysql/updatedatabase.pl
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!
- Now start from a clean DB
- Run:
  $ reset_all
  (y)
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!

Verify you can use emojis all over the place (MARC records, AV
descriptions, etc).

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #70905|0   |1
is obsolete||

--- Comment #29 from Katrin Fischer  ---
Created attachment 71211
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71211=edit
Bug 18336: SET NAMES utf8mb4 in Koha::Database

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

  Attachment #70904|0   |1
is obsolete||

--- Comment #28 from Katrin Fischer  ---
Created attachment 71210
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71210=edit
Bug 18336: Full stack tests for supplemental UTF-8 chars

This patch introduces tests for Koha's support for 4-byte supplemental
UTF-8 chars. encoding/decoding tools handle this gracefuly. The missing
piece is the MySQL DB backend.

The tests in this patch:
- Adds a couple records for each flavour (MARC21 and UNIMARC) so
  search_utf8.t tests 4-byte chars are handled correctly
- Adds emoji testing in auth_values_input_www.t

To test:
- Apply this patch
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: It should fail if the DB hasn't been migrated into using
utf8mb4

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #27 from Katrin Fischer  ---
Need to fix the QA tools?

FAILinstaller/data/mysql/kohastructure.sql
   FAIL   charset_collate
The table columns_settings does not have the current charset
collate (see bug 11944)
...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

 Status|Failed QA   |Signed Off

--- Comment #26 from Katrin Fischer  ---
Without the sample data the tests fail - but I guess that's ok.
Passing QA on that assumption, please stop me if I am mistaken ;)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #25 from Katrin Fischer  ---
Hm it worked on a clean db (of course) :)

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-02-04 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

 Status|Signed Off  |Failed QA

--- Comment #24 from Katrin Fischer  ---
I've tried, but failed:

kohadev-koha@kohadevbox:/home/vagrant/kohaclone$
./installer/data/mysql/updatedatabase.pl 
DEV atomic update: bug_18336_utf8mb4.perl
Upgrade to XXX done (Bug 18336: Convert DB tables to utf8mb4)

kohadev-koha@kohadevbox:/home/vagrant/kohaclone$ prove
t/db_dependent/www/search_utf8.t  
t/db_dependent/www/auth_values_input_www.t
t/db_dependent/www/search_utf8.t  1/99 Link not found at
t/db_dependent/www/search_utf8.t line 146.
# Looks like your test exited with 255 just after 4.
t/db_dependent/www/search_utf8.t  Dubious, test returned 255 (wstat
65280, 0xff00)
Failed 95/99 subtests 
t/db_dependent/www/auth_values_input_www.t .. 3/34 Error POSTing
http://kohadev-intra.myDNSname.org:8080/cgi-bin/koha/admin/authorised_values.pl:
Internal Server Error at t/db_dependent/www/auth_values_input_www.t line 85.
# Looks like your test exited with 255 just after 5.
t/db_dependent/www/auth_values_input_www.t .. Dubious, test returned 255 (wstat
65280, 0xff00)
Failed 29/34 subtests 

Test Summary Report
---
t/db_dependent/www/search_utf8.t  (Wstat: 65280 Tests: 4 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 99 tests but ran 4.
t/db_dependent/www/auth_values_input_www.t (Wstat: 65280 Tests: 5 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 34 tests but ran 5.
Files=2, Tests=9,  7 wallclock secs ( 0.03 usr  0.00 sys +  2.71 cusr  0.80
csys =  3.54 CPU)
Result: FAIL

Did I miss something?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-29 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #23 from Katrin Fischer  ---
(In reply to Tomás Cohen Arazi from comment #20)
> (In reply to Katrin Fischer from comment #19)
> > Wondering, should we also keep the case sensitivity on subfields for the
> > auth frameworks? The item search fields table might be another use case.
> 
> Katrin, possibly. But a separate bug for sure.

Agreed - I just noticed we didn't do it for those tables.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-29 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #22 from Tomás Cohen Arazi  ---
(In reply to Tomás Cohen Arazi from comment #21)
> Created attachment 71022 [details] [review]
> Bug 18336: Fix missing utf8_bin > utf8mb4_bin translation in
> kohastructure.sql
> 
> Signed-off-by: Tomas Cohen Arazi 

This was covered on the atomic update, but I forgot to add it to
kohastructure.sql.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-29 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #21 from Tomás Cohen Arazi  ---
Created attachment 71022
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=71022=edit
Bug 18336: Fix missing utf8_bin > utf8mb4_bin translation in kohastructure.sql

Signed-off-by: Tomas Cohen Arazi 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-29 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #20 from Tomás Cohen Arazi  ---
(In reply to Katrin Fischer from comment #19)
> Wondering, should we also keep the case sensitivity on subfields for the
> auth frameworks? The item search fields table might be another use case.

Katrin, possibly. But a separate bug for sure.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-28 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Katrin Fischer  changed:

   What|Removed |Added

 CC||katrin.fisc...@bsz-bw.de

--- Comment #19 from Katrin Fischer  ---
Wondering, should we also keep the case sensitivity on subfields for the auth
frameworks? The item search fields table might be another use case.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

 Status|Needs Signoff   |Signed Off

--- Comment #18 from M. Tompsett  ---
Followed test plan. Also added a patron with emojis in so many fields,
including username and password! And the patron was able to log in with emojis!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

  Attachment #70855|0   |1
is obsolete||

--- Comment #13 from M. Tompsett  ---
Created attachment 70905
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70905=edit
Bug 18336: SET NAMES utf8mb4 in Koha::Database

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

  Attachment #70856|0   |1
is obsolete||

--- Comment #14 from M. Tompsett  ---
Created attachment 70906
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70906=edit
Bug 18336: Convert schema from utf8 to utf8mb4

This patch adapts the DB structure so it uses utf8mb4 encoding and
utf8mb4_generic_ci collation.

Indexes for columns of type VARCHAR with prefix lenght higher than 191
are shortened because of the smaller max index lenght for utf8mb4

Note: please beware that testing this patchset risks your data and the
patchset includes reinitializing the DB.

To test:
- Be on the master branch
- Have a clean DB:
  $ reset_all
  (y)
- Apply the first patch (Unit tests)
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: Tests fail because Koha doesn't support supplemental (UTF-8)
 chars.
- Apply the rest of this patchset
- Upgrade the schema:
  $ kshell
 k$ perl installer/data/mysql/updatedatabase.pl
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!
- Now start from a clean DB
- Run:
  $ reset_all
  (y)
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!

Verify you can use emojis all over the place (MARC records, AV
descriptions, etc).

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

  Attachment #70857|0   |1
is obsolete||

--- Comment #15 from M. Tompsett  ---
Created attachment 70907
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70907=edit
Bug 18336: DBIC update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

  Attachment #70859|0   |1
is obsolete||

--- Comment #17 from M. Tompsett  ---
Created attachment 70909
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70909=edit
Bug 18336: Add explicit index names to kohastructure.sql

This patch just adds an explicit index name to the search_marc_map
table. The atomicupdate on this patchset is a good example of why we
better have them.

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

  Attachment #70858|0   |1
is obsolete||

--- Comment #16 from M. Tompsett  ---
Created attachment 70908
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70908=edit
Bug 18336: Atomic update

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

  Attachment #70854|0   |1
is obsolete||

--- Comment #12 from M. Tompsett  ---
Created attachment 70904
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70904=edit
Bug 18336: Full stack tests for supplemental UTF-8 chars

This patch introduces tests for Koha's support for 4-byte supplemental
UTF-8 chars. encoding/decoding tools handle this gracefuly. The missing
piece is the MySQL DB backend.

The tests in this patch:
- Adds a couple records for each flavour (MARC21 and UNIMARC) so
  search_utf8.t tests 4-byte chars are handled correctly
- Adds emoji testing in auth_values_input_www.t

To test:
- Apply this patch
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: It should fail if the DB hasn't been migrated into using
utf8mb4

Sponsored-by: Hotchkiss School

Signed-off-by: Mark Tompsett 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #11 from Tomás Cohen Arazi  ---
(In reply to M. Tompsett from comment #10)
> (In reply to M. Tompsett from comment #9)
> > Why did the size change from 255 to 191?
> 
> After a conversation on IRC with tcohen:
> A lot of fields are shrunk from 255 to 191. This is because

A lot of *indexes* are shrunk from 255 to 191. This has the impact of making
searches on things longer than 191 chars require some more CPU usage
(probably). But no data is shrunk in most of the changes.

The tags_* tables get the 'term' column shrunk, because I didn't manage to fix
it other way. It seems a safe one, though. Tags are usually a single word...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

URL||https://dev.mysql.com/doc/r
   ||efman/5.5/en/charset-unicod
   ||e-conversion.html

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

M. Tompsett  changed:

   What|Removed |Added

 CC||mtomp...@hotmail.com

--- Comment #10 from M. Tompsett  ---
(In reply to M. Tompsett from comment #9)
> Why did the size change from 255 to 191?

After a conversation on IRC with tcohen:
A lot of fields are shrunk from 255 to 191. This is because
index size for VARCHAR on InnoDB has a max size, in bytes, of 767 bytes (or
similar) and utf8 uses 3 bytes for each char, and utf8mb4, 4.
767/3 > 255. but 767/4 > 191.
https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-conversion.html

Gain functionality (necessary for true unicode), lose space (not an issue yet).

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-24 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #9 from M. Tompsett  ---
Comment on attachment 70857
  --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70857
Bug 18336: DBIC update

Review of attachment 70857:
 --> 
(https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html=18336=70857)
-

Why did the size change from 255 to 191?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

 Blocks||20080


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20080
[Bug 20080] Test for utf8mb4 encoding and collation
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

 Blocks||15794


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15794
[Bug 15794] Allow the use of emojis as tags on records
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

   See Also|https://bugs.koha-community |
   |.org/bugzilla3/show_bug.cgi |
   |?id=15794   |
 CC||kyle.m.h...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

 Status|NEW |Needs Signoff

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #8 from Tomás Cohen Arazi  ---
Created attachment 70859
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70859=edit
Bug 18336: Add explicit index names to kohastructure.sql

This patch just adds an explicit index name to the search_marc_map
table. The atomicupdate on this patchset is a good example of why we
better have them.

Sponsored-by: Hotchkiss School

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #6 from Tomás Cohen Arazi  ---
Created attachment 70857
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70857=edit
Bug 18336: DBIC update

Sponsored-by: Hotchkiss School

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #7 from Tomás Cohen Arazi  ---
Created attachment 70858
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70858=edit
Bug 18336: Atomic update

Sponsored-by: Hotchkiss School

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #5 from Tomás Cohen Arazi  ---
Created attachment 70856
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70856=edit
Bug 18336: Convert schema from utf8 to utf8mb4

This patch adapts the DB structure so it uses utf8mb4 encoding and
utf8mb4_generic_ci collation.

Indexes for columns of type VARCHAR with prefix lenght higher than 191
are shortened because of the smaller max index lenght for utf8mb4

Note: please beware that testing this patchset risks your data and the
patchset includes reinitializing the DB.

To test:
- Be on the master branch
- Have a clean DB:
  $ reset_all
  (y)
- Apply the first patch (Unit tests)
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: Tests fail because Koha doesn't support supplemental (UTF-8)
 chars.
- Apply the rest of this patchset
- Upgrade the schema:
  $ kshell
 k$ perl installer/data/mysql/updatedatabase.pl
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!
- Now start from a clean DB
- Run:
  $ reset_all
  (y)
- Run the tests:
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> SUCCESS: Tests pass!

Verify you can use emojis all over the place (MARC records, AV
descriptions, etc).

Sponsored-by: Hotchkiss School

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #3 from Tomás Cohen Arazi  ---
Created attachment 70854
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70854=edit
Bug 18336: Full stack tests for supplemental UTF-8 chars

This patch introduces tests for Koha's support for 4-byte supplemental
UTF-8 chars. encoding/decoding tools handle this gracefuly. The missing
piece is the MySQL DB backend.

The tests in this patch:
- Adds a couple records for each flavour (MARC21 and UNIMARC) so
  search_utf8.t tests 4-byte chars are handled correctly
- Adds emoji testing in auth_values_input_www.t

To test:
- Apply this patch
- Run:
  $ kshell
 k$ prove t/db_dependent/www/search_utf8.t \
  t/db_dependent/www/auth_values_input_www.t
=> FAIL: It should fail if the DB hasn't been migrated into using
utf8mb4

Sponsored-by: Hotchkiss School

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #4 from Tomás Cohen Arazi  ---
Created attachment 70855
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=70855=edit
Bug 18336: SET NAMES utf8mb4 in Koha::Database

Sponsored-by: Hotchkiss School

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-23 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Jonathan Druart  changed:

   What|Removed |Added

   Keywords||rel_18_05_candidate

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2018-01-19 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Tomás Cohen Arazi  changed:

   What|Removed |Added

 CC||tomasco...@gmail.com
   Assignee|gmcha...@gmail.com  |tomasco...@gmail.com
  Change sponsored?|--- |Sponsored

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-04-06 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Jonathan Druart  changed:

   What|Removed |Added

 CC||jonathan.dru...@bugs.koha-c
   ||ommunity.org
 Depends on||11944


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11944
[Bug 11944] Cleanup Koha UTF-8
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-04-06 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Jonathan Druart  changed:

   What|Removed |Added

 Depends on|11944   |


Referenced Bugs:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11944
[Bug 11944] Cleanup Koha UTF-8
-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-04-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Marc Véron  changed:

   What|Removed |Added

 CC||ve...@veron.ch

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-04-05 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Josef Moravec  changed:

   What|Removed |Added

 CC||josef.mora...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-03-31 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

David Cook  changed:

   What|Removed |Added

   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=11944

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-03-31 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

David Cook  changed:

   What|Removed |Added

   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=15794

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-03-26 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #2 from Martin Renvoize  ---
We (PTFS Europe) have indeed been  installing using utf8mb4 as standard
practice for a good number of years now and can confirm all of what you've said
is true, and that we've never seen any side effects.

I'd certainly back it's use during installation by default.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-03-26 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

--- Comment #1 from David Cook  ---
Interestingly DBD::mysql actually recommends using mysql_enable_utf8mb4 over
mysql_enable_utf8 http://search.cpan.org/dist/DBD-mysql/lib/DBD/mysql.pm. It
looks like we might be calling SET NAMES redundantly in Koha::Database as well,
but I don't think that matters too much.

You can use the following query to see if you have any incomplete MARCXML
records in Koha:

select biblionumber,metadata from biblio_metadata where
ExtractValue(metadata,'/record') is null;

Actually, that query is useful for finding any MARCXML record that will cause a
XML parser to fail.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-03-26 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

Mason James  changed:

   What|Removed |Added

 CC||m...@kohaaloha.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 18336] Use utf8mb4 instead of utf8 for MySQL tables, columns, and connections

2017-03-26 Thread bugzilla-daemon

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18336

David Cook  changed:

   What|Removed |Added

 CC||martin.renvoize@ptfs-europe
   ||.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

92 matches

Mail list logo