• Non in-place upgrade of MySQL 5.7 to 8

    From hyjinx@1337:2/104 to All on Mon Jul 19 16:46:46 2021
    Hi All,

    I'm building a new box. It was about time I got my A-into-G. My current webserver runs Ubuntu 18.04, and whilst it's still LTS supported for a couple of more years, running PHP 7.2 irks me, plus it's a nice chance to migrate
    the box from Germany to USA.

    ãSo, I've built a new box running Ubuntu Server 20.04. Grand. It comes with MySQL 8 out of the box. It seems I've been asleep for a lot longer than I thought, because Ubuntu 18.04 runs MySQL 5.7.34, and there is no 'simple' upgrade path. A lot of things have changed, so my usual:
    mysqldump --all-databases > foo.sql won't cut the mustard. If I import that into MySQL 8, it tells me that it hates the mysql table and gives up, can't even run the mysql_upgrade tool. After a few hours of googling, this is a
    well known matter (thanks Oracle), but I can't seem to find anyone that suggests how I can migrate the databases WITHOUT having to set up all the accounts manually again (i.e., create new mysql users and set new passwords). This is pretty much a no go; I'd have to reach out to my clients and get them to give me new passwords and then reconfigure their websites to use the new passwords. I see much breakage and annoyed customers calling me.

    Surely, there is a way I can do some scripting or some tool can automate the migration? Any ideas?

    Cheers,
    Al


    hyjinx // Alistair Ross
    Author of 'Back to the BBS' Documentary: https://bit.ly/3tRINeL (YouTube) alsgeeklab.com

    --- Mystic BBS v1.12 A46 2020/08/26 (Linux/64)
    * Origin: Sent from Al's Geek Lab 1337:2/104 (1337:2/104)
  • From MeaTLoTioN@1337:1/101 to hyjinx on Mon Jul 19 07:17:57 2021
    On 19 Jul 2021, hyjinx said the following...

    Hi All,

    Hey hyjinx =)

    Surely, there is a way I can do some scripting or some tool can automate themigration? Any ideas?

    Have a look at this; https://dba.stackexchange.com/questions/282215/migrate-mysql-5-7-to-8-0-via-mys qldump

    TLDR;

    Recreate the dump from MySQL 5.7
    Personally, I would recreate the dump without mysql schema.

    SQL="SET group_concat_max_len = 1024 * 1024;"
    SQL="${SQL} SELECT GROUP_CONCAT(schema_name separator ' ')"
    SQL="${SQL} FROM information_schema.schemata WHERE schema_name NOT IN" SQL="${SQL} ('information_schema','performance_schema','mysql','sys')" DBLIST=`mysql -u... -p... -AN -e"${SQL}"`
    mysqldump -u... -p... -B ${DBLIST} > mydbs.sql
    Then, you can load mydbs.sql in the MySQL 8.0 instance.


    What about the dump you already have ?


    First, find out the lines in the dump where each database starts and the
    number of line in the dump

    grep -in "^USE" mydbs.sql
    wc -l < mydbs.sql

    As an example, I created a dump from a MySQL 5.7 vagrant instance.

    root@LAPTOP-57FSQ99U:~# grep -in "^USE" mydbs.sql
    24:USE `DMP492`;
    122:USE `DMP551`;
    190:USE `DMP579`;
    228:USE `abeersul`;
    262:USE `dbname`;
    292:USE `karma`;
    300:USE `mysql`;
    1212:USE `pch_prod`;
    1220:USE `pchcom_prod`;
    1286:USE `tryout`;
    root@LAPTOP-57FSQ99U:~# wc -l < mydbs.sql
    1341
    root@LAPTOP-57FSQ99U:~#

    Next, create a new dump. Copy everything above and below the mysql schema
    In this case, everything before line 300

    Then, copy everything from the first DB after mysql schema
    In this case, subtract 1212 (first line of the next DB after the mysql schema) from 1341 (the number of lines in the original dump you made) and add

    1. Then copy that number of lines from the bottom of the original dump into
    the new dump:

    head -299 mydbs.sql > newdump.sql
    (( LINES = 1341 - 1212 + 1 ))
    tail -${LINES} mydbs.sql >> newdump.sql
    As a demonstration, note how the new dump will not have the mysql schema

    root@LAPTOP-57FSQ99U:~# head -299 mydbs.sql > newdump.sql root@LAPTOP-57FSQ99U:~# (( LINES = 1341 - 1212 + 1 ))
    root@LAPTOP-57FSQ99U:~# tail -${LINES} mydbs.sql >> newdump.sql root@LAPTOP-57FSQ99U:~# grep -in "^USE" newdump.sql
    24:USE `DMP492`;
    122:USE `DMP551`;
    190:USE `DMP579`;
    228:USE `abeersul`;
    262:USE `dbname`;
    292:USE `karma`;
    300:USE `pch_prod`;
    308:USE `pchcom_prod`;
    374:USE `tryout`;
    root@LAPTOP-57FSQ99U:~#

    Now, just load newdump.sql into MySQL 8.0


    Hope this helps

    ---
    |14Best regards,
    |11Ch|03rist|11ia|15n |11a|03ka |11Me|03aTLoT|11io|15N

    |07ÄÄ |08[|10eml|08] |15ml@erb.pw |07ÄÄ |08[|10web|08] |15www.erb.pw |07ÄÄÄ¿ |07ÄÄ |08[|09fsx|08] |1521:1/158 |07ÄÄ |08[|11tqw|08] |151337:1/101 |07ÂÄÄÙ |07ÄÄ |08[|12rtn|08] |1580:774/81 |07ÄÂ |08[|14fdn|08] |152:250/5 |07ÄÄÄÙ
    |07ÄÄ |08[|10ark|08] |1510:104/2 |07ÄÙ

    --- Mystic BBS v1.12 A47 2020/12/04 (Linux/64)
    * Origin: thE qUAntUm wOrmhOlE, rAmsgAtE, uK. bbs.erb.pw (1337:1/101)
  • From hyjinx@1337:2/104 to MeaTLoTioN on Mon Jul 19 18:48:46 2021
    Thanks dude. Unfortunately that link is one of the many links I already read.

    The main problem is this bit:
    Personally, I would recreate the dump without mysql schema.

    That's fine - but the main problem is how do I Migrate the users across? Dumping the databases sans the mysql database is easy enough, but I still
    need the user from mysql.users at the very least, right?

    Cheers,
    Al


    hyjinx // Alistair Ross
    Author of 'Back to the BBS' Documentary: https://bit.ly/3tRINeL (YouTube) alsgeeklab.com

    --- Mystic BBS v1.12 A46 2020/08/26 (Linux/64)
    * Origin: Sent from Al's Geek Lab 1337:2/104 (1337:2/104)
  • From MeaTLoTioN@1337:1/101 to hyjinx on Mon Jul 19 08:57:49 2021
    On 19 Jul 2021, hyjinx said the following...

    Thanks dude. Unfortunately that link is one of the many links I already read.

    The main problem is this bit:
    Personally, I would recreate the dump without mysql schema.

    That's fine - but the main problem is how do I Migrate the users across? Dumping the databases sans the mysql database is easy enough, but I still need the user from mysql.users at the very least, right?

    Hmm, wouldn't the users be in the .sql file though? only the schema wouldn't, no?

    In the example, he's grabbing the mysql db along with the others, and the users

























































































































































































































































































































































































































































































































































































































































































































































    table is within this db, so I would have thought it would include the users data?

    ---
    |14Best regards,
    |11Ch|03rist|11ia|15n |11a|03ka |11Me|03aTLoT|11io|15N

    |07ÄÄ |08[|10eml|08] |15ml@erb.pw |07ÄÄ |08[|10web|08] |15www.erb.pw |07ÄÄÄ¿ |07ÄÄ |08[|09fsx|08] |1521:1/158 |07ÄÄ |08[|11tqw|08] |151337:1/101 |07ÂÄÄÙ |07ÄÄ |08[|12rtn|08] |1580:774/81 |07ÄÂ |08[|14fdn|08] |152:250/5 |07ÄÄÄÙ
    |07ÄÄ |08[|10ark|08] |1510:104/2 |07ÄÙ

    --- Mystic BBS v1.12 A47 2020/12/04 (Linux/64)
    * Origin: thE qUAntUm wOrmhOlE, rAmsgAtE, uK. bbs.erb.pw (1337:1/101)
  • From deon@1337:2/101 to hyjinx on Mon Jul 19 17:58:35 2021
    Re: Non in-place upgrade of MySQL 5.7 to 8
    By: hyjinx to All on Mon Jul 19 2021 04:46 pm

    Surely, there is a way I can do some scripting or some tool can automate
    the migration? Any ideas?

    Do you have to use MySQL? Why not switch to MariaDB?

    I pretty much stopped using MySQL a long time ago (when Oracle bought it) and using MariaDB for any MySQL requirements.

    (Plus I run everything in docker, so its easy to move stuff around, or revert back if something breaks...)

    ...ëîåï

    ... When your work speaks for itself, don`t interrupt.
    --- SBBSecho 3.14-Linux
    * Origin: I'm playing with ANSI+videotex - wanna play too? (1337:2/101)
  • From MeaTLoTioN@1337:1/101 to hyjinx on Mon Jul 19 09:01:21 2021

    On 19 Jul 2021, MeaTLoTioN said the following...

    On 19 Jul 2021, hyjinx said the following...

    Thanks dude. Unfortunately that link is one of the many links I alrea read.

    The main problem is this bit:
    Personally, I would recreate the dump without mysql schema.

    That's fine - but the main problem is how do I Migrate the users acro Dumping the databases sans the mysql database is easy enough, but I s need the user from mysql.users at the very least, right?

    Hmm, wouldn't the users be in the .sql file though? only the schema wouldn't,no?

    In the example, he's grabbing the mysql db along with the others, and
    the userstable is within this db, so I would have thought it would
    include the usersdata?


    Unless I actually _read_ the last line...

    "To clarify, when I've completed the restore I'll have to create users in MYSQL

























































































































































































































































































































































































































































































































































































































































































































































    8.0 to match the original users in the MYSQL5.7 database?"

    I think that's exactly what you said, so now I'm at a loss.

    ---
    |14Best regards,
    |11Ch|03rist|11ia|15n |11a|03ka |11Me|03aTLoT|11io|15N

    |07ÄÄ |08[|10eml|08] |15ml@erb.pw |07ÄÄ |08[|10web|08] |15www.erb.pw |07ÄÄÄ¿ |07ÄÄ |08[|09fsx|08] |1521:1/158 |07ÄÄ |08[|11tqw|08] |151337:1/101 |07ÂÄÄÙ |07ÄÄ |08[|12rtn|08] |1580:774/81 |07ÄÂ |08[|14fdn|08] |152:250/5 |07ÄÄÄÙ
    |07ÄÄ |08[|10ark|08] |1510:104/2 |07ÄÙ

    --- Mystic BBS v1.12 A47 2020/12/04 (Linux/64)
    * Origin: thE qUAntUm wOrmhOlE, rAmsgAtE, uK. bbs.erb.pw (1337:1/101)