MySQL deployment issue on IBM Cloud’s Managed Kubernetes Service due to lost+found directory




MySQL Statefulset manifest

but I got the following errors:

2019-08-25T15:36:470279Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting2019-08-09T15:36:470367Z 0 [ERROR] Aborting 

After debugging the issue I found out that:

mysql container’s entrypoint script checks if DATADIR/mysql (in my case it was /var/lib/mysql) exists. If not, it run --initialize, but if DATADIR/mysql contains any files other than ones starting with . or specified with --ignore-db-dir, --initialize will fail with the above error message.

The error was due to the lost+found directory that is being added in DATADIR/mysql directory when a volume is mounted and it is the default behaviour according to IBM Cloud support.

What purpose does lost+found serve?

A file element contains one or more versions of a file. A directory element contains one or more versions of a directory, each of which can contain file elements and other directory elements.

Versions of text files are distinguished by changes in their text. Versions of directories are distinguished by changes in their contents (to account for files and subdirectories that have been renamed, removed, or added).

Every VOB includes a special directory element, lost+found, which is used to hold elements that become stranded when they are not catalogued in any directory version in the VOB.

An element can become stranded when you do any of the following:

  • Create new elements, and then cancel the checkout of the directory in which they were created
  • Delete the last reference to an element by using the rmname command
  • Delete the last reference to an element by deleting a directory version with the rmver, rmbranch, or rmelemcommand


Init container manifest

Init container removes the lost+found directory before the MySQL server start inside the container.

Final manifest is given below:

Final MySQL manifest for IBM Cloud

MySQL Data Backup

To backup MySQL data on AWS S3 bucket follow the guidelines provided in this story.

Final Thoughts

DevSecOps Engineer

DevSecOps Engineer