Find missing files in a folder of sequentially numbered files

The Bash script missing_files finds missing files in a folder of sequentially numbered files, such as the frames from a film digitisation. The program is part of Bash AVpres, released under a 3-Clause BSD License and provided “as is” without warranty or support of any kind.

Documentation

A short help message is embedded into the script:

$ missing_files -h

Usage:
  missing_files -i <input_folder> [-o <output_file>]
  missing_files -h | -x
Options:
  -i  folder to check
  -o  result file (default is '<folder>_missing.txt')
  -h  this help
  -x  advanced options with their default arguments
See also:
  man missing_files
  https://avpres.net/Bash_AVpres/
About:
  Abstract: Find missing files in a folder of sequentially numbered files.
  Version:  2023-03-26

$

A detailed manual page is installed together with the script (man missing_files) and is available also here as a PDF file on this website.

Example

Create a folder and some sequentially numbered files in it

$ mkdir TEST_ORIG
$ ffmpeg -lavfi testsrc2 -t 10 -pix_fmt rgb48 -start_number 86400 TEST_ORIG/%06d.tif
$

Make a copy of the original folder and verity that all the files are included

$ cp -R TEST_ORIG TEST_COPY
$ missing_files -i TEST_COPY
All files are present in 'TEST_COPY'.
$

Delete five randomly chosen files from the copy folder and verify again

$ find TEST_COPY -type f | sort -R | tail -n 5 | while read f; do rm $f; done
$ missing_files -i TEST_COPY
TEST_COPY/086467.tif is missing.
TEST_COPY/086559.tif is missing.
TEST_COPY/086615.tif is missing.
TEST_COPY/086616.tif is missing.
TEST_COPY/086639.tif is missing.
$

Use the generated text file to complete the copy folder and verify a last time

$ for f in $(cat TEST_COPY_missing.txt); do cp -v "TEST_ORIG/$f" "TEST_COPY"; done
TEST_ORIG/086467.tif -> TEST_COPY/086467.tif
TEST_ORIG/086559.tif -> TEST_COPY/086559.tif
TEST_ORIG/086615.tif -> TEST_COPY/086615.tif
TEST_ORIG/086616.tif -> TEST_COPY/086616.tif
TEST_ORIG/086639.tif -> TEST_COPY/086639.tif
$ missing_files -i TEST_COPY
All files are present in 'TEST_COPY'.
$

The details can be retrieved from the log files

$ ls -l /tmp/AVpres
total 24
-rw-------  1 test  wheel  259 18 Sep 17:30 missing_files.BcWUAXFpc1
-rw-------  1 test  wheel  259 18 Sep 17:37 missing_files.TJqRxdtxgj
-rw-------  1 test  wheel  434 18 Sep 17:33 missing_files.g4KkCFJoPP
$ cat /tmp/AVpres/missing_files.g4KkCFJoPP
[2020-09-18 15:33:50 UTC] missing_files 2020-09-19
[2020-09-18 15:33:50 UTC] /usr/local/bin/missing_files -i TEST_COPY
[2020-09-18 15:33:50 UTC] START
[2020-09-18 15:33:50 UTC] verify input
[2020-09-18 15:33:50 UTC] verify output
[2020-09-18 15:33:50 UTC] check files in folder
The following files of the intervall from '086400' to '086649' are missing:
  086467.tif
  086559.tif
  086615.tif
  086616.tif
  086639.tif
[2020-09-18 15:33:51 UTC] END
$

2023-03-26