This is surprisingly not intuitive, as you cannot simply pass an array of integers to a prepared statement. Instead, you need to add special binding types to inform Doctrine that this is indeed an array of integers.

It’s very well explained in the documentation under the Parameters Conversion section:
https://www.doctrine-project.org/projects/doctrine-dbal/en/current/reference/data-retrieval-and-manipulation.html#list-of-parameters-conversion
But it is rather hard to find and it took me a while.

The trick is to add the array binding types to the the types parameters to the query method. In the case of integers, it is \Doctrine\DBAL\ArrayParameterType::INTEGER.
Now Doctrine knows how to handle the types in the array while binding the values.

Example:

$sql = 'SELECT * FROM items WHERE id IN (?)'; 
$numbersAsStrings = "1,2,3";
$numbers = array_map('intval', explode(',', $numbersAsStrings));
$users = $con->fetchAllAssociative(
  $sql, 
  [$numbers], 
  [ArrayParameterType::INTEGER]
);

If you omit the types, Doctrine will simply try to bind the array as string, which will fail and you get this error:
Array to string conversion

If you try to pass it as a comma-separated string yourself and omit the types like $numbers = implode(',',array_map('intval', explode(',', $numbersAsStrings))); You will get an error because Doctrine expects an array of values for the IN clause and not a string.
Error:
count(): Argument #1 ($value) must be of type Countable|array, string given

I hope this helps someone finding out how to make WHERE IN SQL queries with Doctrine more quickly than me next time.

my_db.mysql      | 2024-08-16T09:40:21.463770Z 2 [ERROR] [MY-010520] [Server] Invalid (old?) table or database name 'mysql.sock'
my_db.mysql      | 2024-08-16T09:40:21.468287Z 2 [ERROR] [MY-010784] [Server] Failed to open dir /var/lib/mysql/mysql.sock

Apparently, MySQL 8 treats every file or directory in the data directory as a database table, whereas in the previous version, MySQL 5, only directories were recognized as database tables.

As the the changelog of MySql 8 states:

Because the data dictionary provides information about database objects, the server no longer checks directory names in the data directory to find databases. Consequently, the --ignore-db-dir option and ignore_db_dirs system variable are extraneous and have been removed. Update system configurations and application programs accordingly.

As a result, all files in the data directory volume, in addition to the directories, are mistakenly recognized as database tables–for example, the mysql.sock file, which is created by Docker.

The solution is to separate the actual database tables from the metadata files. We need to move all table files to a dedicated data directory and configure MySQL to use this new location.
Then let’s move all database data to dedicated data dir:

mkdir data
mv * ./data/

And specify the dataDir in my.cnf:
datadir=/var/lib/mysql/data

Now MySql will only look for database table files in this directory and Docker still can write to the main MySql directory.
And, most importantly, the database will start up again.

There are several ways to import the data in your docker-compose setup.

1. Using a volume for import data
2. Using mysql client from commandline with docker-compose exec
3. Using phpmyadmin in docker-compose setup
4. Using a mysql GUI client on the host and connect to the DB in the Docker container

So let’s see how we can do this:

1. Using a volume for import data

This applies to both official Mysql and MariaDB official images.
These have builtin import mechanism to import data.
To quote from the docs:

Furthermore, it will execute files with extensions .sh, .sql and .sql.gz that are found in /docker-entrypoint-initdb.d. Files will be executed in alphabetical order. You can easily populate your mysql services by mounting a SQL dump into that directory

So all dumps that are found in the /docker-entrypoint-initdb.d directory of the image will be imported unless the database already contains data.

So let’s add this volume to our docker-compose.yaml

volumes: - ./database_dump.sql:/docker-entrypoint-initdb.d/datadump.sql

On next docker-compose up the data will be imported in your empty database.
You will see something like this in your docker-compose logs:

2023-02-06 15:40:16+00:00 [Note] [Entrypoint]: /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/database_dump.sql

Note: if there is already data in the database you need to clear the data first by f.e. clearing the volume, otherwise the import will not start:
You can clear all volumes of your docker-compose setup with (caution: this will clear all volumes, not just the database volume):

docker-compose down -v

2. Using mysql client from commandline with docker-compose exec

With this one liner you can import a SQL dump from a docker image that has a MySql/MariaDB client installed and are linked to the DB container:

docker-compose exec my-app bash -c "mysql -u root -h mysql --password=root database < database_dump.sql"

This will execute the application image and import the data with the mysql client which connects to the host mysql which is the linked mysql container.

3. Use phpmyadmin in docker-compose setup

If the above is too consolish then you also can just add a phpmyaadmin container in your docker-compose setup and administer the database with a GUI from the browser.

  myproject_phpmyadmin:
	image: phpmyadmin/phpmyadmin:latest
	ports:
	  - 8080:80
	environment:
	  PMA_HOST: myproject_mysql

Then you can just open http://localhost:8080 in the browser and log into to Phpmyadmin.
To import data go to the import tab and upload the dump file.

4. Using a mysql GUI client on the host and connect to the DB in the Docker container

For this approach you just need to open a port in the database container to the outside and use this port with localhost in the settings of your database client.

  myproject_mysql:
    image: mysql:5.7
    hostname: mysql.${DOMAIN}
    container_name: ${CONTAINER_NAME}.mysql
      volumes:
	- ${PWD}/data/mysql/:/var/lib/mysql
	- ./container/mysql/my.cnf:/etc/mysql/conf.d/z_my.cnf
    ports: # open the mysql port to the host
      - "3306:3306"
    environment:
  ...

See this screenshot from IntellIj’s PHPStorm Database tool:

From my experience these clients are very good to check data but not so good for import, at least I had rather poor experience in the PHPStorm DB Tool.
So going consolish on imports is recommended :)

An exception occurred while executing 'INSERT INTO user (id, email, roles, password, is_verified) VALUES (?, ?, ?, ?, ?)' with params [3, "dev@dev.de", "[]", "your-encrypted-password", 0]:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near "user"
LINE 1: INSERT INTO user (id, email, roles, password, is_verified) V...

To fix this you have to escape the table name on your entity, fe. User.php:

@ORM\Table(name="`user`")

(note the backticks inside the quotes!)

If you generate the entity with maker bundle: bin/console make:entity directly on Postgres the backticks are added automatically.
But not when you switch the DB type. Then you have to add them manually. :)

A Sqlite database is only one file, which you surely have access to.
So just add a Database configuration to your IDE and point it to the Sqlite file.
Click on the Database flyout on the left side then on the + (Plus) and add a Sqlite configuration.
(IntelliJ has good documentation on Sqlite Database connection as well of course)

Then you are ready to go.
The UI of the database structure is quite good.
You can browse the tables by doublelick on the table in the database structure view.
Also you can edit values in the table view.
Note: When the DB was created inside the Docker container you probably dont own the DB from the host and you wont be able write to it.
While developing just change permissions on the host, if necessary.

You can also open a Database Console window where you can test queries.
This is particular better than using Adminer because these test queries are persisted and will still be there the next day.
You can add multiple database consoles and reopen them from “Scratches and Consoles” in your Project menu.

So no Adminer needed. Sqlite is just a file, dude.
No hassles with docker-compose anymore for DB administration and still using Docker for the app. :)

Update 09.08.2023:
JetBrains has now also uploaded a nice Youtube Video Tutorial on how to deal with Sqlite databases in IntelliJ IDEs like f.e. PHPStorm:
Working with SQLite Databases in any JetBrains IDE

I learned that you can now Drag and Drop Sqlite database files into the DataBase Pane and the IDE connects it directly, nice!

Unknown MySQL server host <mysql-service-name>

… and you dont know why because everything seems to be correct.

Then you might have an upgrade problem with mysql because you are reusing an old volume that was created for another mysql version.
This can happen when you use a unspecified tag as mysql:latest (not recommended anyway) and there was a version bump in the official mysql image.
Or you upgraded the mysql container yourself, f.e. from mysql:5.6 to mysql:5.7 and you are reusing the data volume with the mysql files.

When you check the startup logs you might see that mysql container is shutting down after startup because of an error like this:

Can't open the mysql.plugin table. Please run mysql_upgrade to create it.

You can run the upgrade command in the container:

docker exec -it mysql_container_name bash -c "mysql_upgrade -uroot -proot"

Or simply delete the volume and recreate it.
Note: the data will unfortunatly be lost. So better dump the data before deleting the volume and reimport it.
In my case its no problem because im starting a new project anyway.

To delete the volume you can:

docker volume rm <volume id>

Or just delete the data dir:

rm -rf ./data/mysql

Then restart your setup so the mysql container recreates the databases:

docker-compose up

and mysql is working again and the application is able to connect. :)

Update: Today i would rather use the IntelliJ Database Tool for Sqlite administration.

But since Adminer is a popular hacking target they introduced a feature that does not allow to run adminer without a password, out of the box.
Sqlite database usually runs without password and dang, workaround needed!

So we need to add a plugin to adminer that allows password-less login and extend the official Adminer Docker image to include the plugin.

We add a script that loads the official password-less-login plugin and copy it to the plugins-enabled folder of the Adminer image

login-password-less.php:

<?php
require_once('plugins/login-password-less.php');

/** Set allowed password
 * @param string result of password_hash
 */
return new AdminerLoginPasswordLess(
    $password_hash = password_hash("admin", PASSWORD_DEFAULT)
);

Dockerfile:

FROM adminer
USER root
COPY login-password-less.php /var/www/html/plugins-enabled/login-password-less.php
#USER adminer # we run as root because of permissions problems on db file with the volume

In docker-compose.yml:

version: "3"
services:
  app:
    build: ./php
      - mailcatcher
    volumes:
      - "../:/app:rw"
      - "./php/cli/php.ini:/etc/php/7.3/cli/php.ini:ro"
      - "./php/fpm/php.ini:/etc/php/7.3/fpm/php.ini:ro"

  nginx:
    build: ./nginx
    depends_on:
      - app
    command: /bin/sh -c "nginx -g 'daemon off;'"
    volumes:
      - "..:/app:rw"
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
    ports:
      - '8088:80'

  adminer:
    build: ./adminer
    restart: unless-stopped
    volumes:
      - "..:/app:rw"
    ports:
      - 8080:8080

File tree:

• adminer
  – Dockerfile
  – login-password-less.php
• nginx
  …
• php
  …
• docker-compose.yml

Note that since docker-compose.yml version 3 the volumes_from directive was removed in favor of top level volumes to share volumes across images. However this implementation is imho rather weird, so i jut duplicated the volumes declaration in the App container and Adminer container.
Feels rather wrong but thats all i could come up with. Any advice here is welcome.

Now that this is done we can open the Sqlite3 database with the pseudo-password “admin” and manage the database.

After all you could also just copy the one-file Adminer with plugin to the php container and run it from within the php container. Just adjust your Nginx config to allow calling the file directly and take care to not deploy Adminer to live.
There is also a project that bundles the plugin with Adminer to one file: https://github.com/FrancoisCapon/LoginToASqlite3DatabaseWithoutCredentialsWithAdminer/blob/master/build-adminer-4-sqlite3-into-one-file.sh

I will show this with my Superleansilexplate and will integrate it there as an example.
Since i dont want to integrate MongoDB in Superleansilexplate it will just become an additional gist.

Given you have some smaller amount of data like a counter that needs to be stored or other loose coupled datasets, we simply speak to MongoDB “directly” and store the data via Doctrine MongoDB Abstraction Layer.
Since i presume the Data / Document Structure isnt that complex we dont use Doctrine MongoDB ODM (the Object Document Mapper).
If you want to use it instead, try this Silex Extensions.

So we use this SilexExtensions collection and install it via git:

 cd mysilexproject
 git clone git@github.com:fate/Silex-Extensions.git vendor/silex-extension

Then we install the doctrine mongodb libary manually:

 git clone --recursive https://github.com/doctrine/mongodb vendor/mongodb

The extension collection has some more poviders but you can just ignore them and just focus on the MongoDbExtension.

– i actually dont like extensions collections. i would prefer to clone each extensions individually. thou its not much code overhead, but when it comes to the vendor dependencies it can become quite messy IMHO –

Alright now the code.
Edit src/app.php and register the namespace for the extension:

 $app['autoloader']->registerNamespace('SilexExtension', __DIR__ . '/../vendor/silex-extension/src');

Then register the MongoDbExtension;

 $app->register(new SilexExtension\MongoDbExtension(), array(
 'mongodb.class_path' => __DIR__ . '/../vendor/mongodb/lib',
 'mongodb.connection' => array(
 'server' => 'mongodb://mysecretuser:mysecretpassw@localhost',
 'options' => array(),
 'eventmanager' => function($eventmanager) {
 }
 )
 ));

and finally query your MongoDB collection of choice and display the resultset as json:

$app->get('/data-from-mongodb', function() use($app) {

$coll = $app['mongodb']->selectDatabase('mydb')->selectCollection('mycoll');
 $query = array(
 'query' => 'value'
 );
 $sort = array(
 'count' => -1,
 );
 $r = $coll->find($query)->sort($sort)->toArray();
 $response = new Response();
 $response->headers->set('Content-type', 'application/json');
 $response->setContent(json_encode($r));
 return $response;
 });

Easy

Cannot allocate memory at ...

So first i suspected mongodb to use up loads of memory as top showed.

But after some recherche work i learned mongodb only -seems- to use a lot of memory.
see here and here and here
The actual usage was around 20mb RAM, so mongodb was innocent.

The true RAM monsters were some apache and php-fpm zombies, but thats another story.

While suspecting mongodb i thought about outsourcing the mongodb and i found a free and sufficient offer in mongolab.
My interests were on and i gave it a try.
The free version has a limit for up to 240MB storage and since my app is just a small counter it should last for some time.

The registration was easy, creation of the database aswell and the administration panel is also pretty cool and informative.

Then create a user and migrate:

First get my data from the vserver on the vserver:

 mongodump --host 127.0.0.1

Then fetch it to my local machine:

 scp -r me@vserver.de:/var/www/vhosts/verrueckte/dump/mongodumpdata .

Ups i didnt had no mongo client on my local machine, go get it:

 sudo apt-get install mongodb-clients

Now connect to the mongolab via your console

 mongo db123xyz.mongolab.com:12345/thedb -u <user> -p <pw>

Hey works, great!
Alright see you later.

 > exit;

Now import the dumps to continue the counter there were i stopped.

 cd mongodumpdata
 mongorestore -h db123xyz.mongolab.com:12345 -d thedb -u -p .

Good!

Then teach the php app the new connection:

 $this->host = 'mongodb://user:pw@db123xyz.mongolab.com:12345/thedb';
 new mongo($this->host);

There you go! The counter is storing its stuff to the mongolab database.
And i can stop the mongod on my machine and free some memory.

 ps aux | grep mongodb
 kill <PID>

Schaut man sich den generischen EntityUserProvider an, so sieht man ab Zeile 46 sowas wie

    /**
     * {@inheritdoc}
     */
    public function loadUserByUsername($username)
    {
        if (null !== $this->property) {
            $user = $this->repository->findOneBy(array($this->property => $username));
        } else {
            if (!$this->repository instanceof UserProviderInterface) {
                throw new InvalidArgumentException(sprintf('The Doctrine repository "%s" must implement UserProviderInterface.', get_class($this->repository)));
            }

            $user = $this->repository->loadUserByUsername($username);
        }

        if (null === $user) {
            throw new UsernameNotFoundException(sprintf('User "%s" not found.', $username));
        }

        return $user;
    }

Dieser Code besagt, dass nur dann, wenn die Konfigurationeinstellung “property” aus der security.yml entfernt wird (steht üblicherweise auf “username” und gibt das Datenbankattribut an, das eben den Benutzernamen hält), das entsprechende Doctrine-Entity “User” auch eine Repository-Class hat und diese darüber hinaus UserProviderInterface implementiert, man seine eigene loadUserByUsername()-Methode implementieren kann, die dann von der Firewall zur Identifizierung des Benutzer beim Login herangezogen wird (dieser Vorgang ist komplett intransparent, Symfony Magic).

Also aus seiner “alten” security.yml

    providers:
        default:
            users:
                user:  { password: userpass, roles: [ 'ROLE_USER' ] }
                admin: { password: adminpass, roles: [ 'ROLE_ADMIN' ] }
            entity: { class: DvlpCoreBundleEntityUser, property: username }

sowas machen:

     ...
            entity: { class: DvlpCoreBundleEntityUser }

Dann muss das zu User gehörtige UserRepository nur noch UserProviderInterface implementieren, bspw. so:

<?php

namespace DvlpCoreBundleEntity;

use DoctrineORMEntityRepository;
use SymfonyComponentSecurityCoreUserUserProviderInterface;
use SymfonyComponentSecurityCoreUserUserInterface;

/**
 * UserRepository
 *
 * This class was generated by the Doctrine ORM. Add your own custom
 * repository methods below.
 */
class UserRepository extends EntityRepository implements UserProviderInterface
{
    /**
     * {@inheritdoc}
     */
    public function loadUserByUsername($username)
    {
        return $this->getEntityManager()
            ->createQuery('SELECT u FROM DvlpCoreBundle:User u JOIN u.Profile p WHERE u.username = :username OR p.email = :username')
            ->setParameters(array(
                'username' => $username
            ))
            ->getOneOrNullResult();
    }

    /**
     * {@inheritDoc}
     */
    public function refreshUser(UserInterface $user)
    {
        return $this->loadUserByUsername($user->getUsername());
    }

    /**
     * {@inheritDoc}
     */
    public function supportsClass($class)
    {
        // NEVER CALLED ...
        return $class === 'DvlpCoreBundleEntityUser';
    }
}
}

Schon kann man als Benutzernamen entweder den Username oder die E-Mail-Adresse des Users angeben.