Found out how to fix the issue (or how to convert the cp1252 data stored as UTF8, as UTF8MB3).
Note that it will NOT work if you have a mixed cp1252/UTF8 database. It was my case for 8 lines related to the same activity in history table, I don't know how this happened but the activity didn't exist anymore so I just deleted the lines.
This is a proof of concept and should only be used by advanced users on a dev/pre-prod environment before going production.
1) Export your database with mysqldump command line. You should get a file in UTF8.
2) Open the .sql file with Notepad++ or a software which is able to convert/encode from/to CP1252/ANSI and UTF8.
3) In "Encode" menu, you should see "Encode as UTF-8 (without BOM)" ticked.
4) Tick "Convert to ANSI". Nothing should have changed.
5) Tick "Encode as UTF-8 (without BOM)". Magic, all your characters should be converted.
6) $enforceUTF8=true; in ProjeQtOr config file.
7) Run the .sql file on your database or copy the contents of the .sql file to your query window and execute. It will fail if you had UTF8 characters before converting in your database. MySQL will tell you which lines fail.

Test this database on your dev/pre-prod environment.
So far so good during my tests.