File deletions occasionally fail in Dropbox

Observed when processing files with a Python script of my own, but likely to be a nuisance to developers of any software, that allows you to store files in arbitrary locations.

A common pattern when modifying files is to first write them to a temporary file in the same directory (ensuring that it is on the same physical drive) and then replacing the original file with the modified file. This pattern gives safety against data corruption, if the program fails while creating the new version of the file.

In Python, that translates in the simplest case to:

tempfile = file.with_suffix(file.suffix + ".tmp")
tempfile.write_text("hello world")
tempfile.replace(file)

On Windows, I find that this pattern frequently fails with an error like

PermissionError: [WinError 32] The process cannot access the file
because it is being used by another process: 'E:\\Dropbox\\tmp\\test.txt'

but only for files in Dropbox. This means that any script I write must allow for retrying the .replace() step.

To give a more complete, read-to use, example I provide the subsequent script, which repeatedly performs the pattern both in %TEMP% and in E:\Dropbox\tmp. On my System, Dropbox is on E:\Dropbox, so that part has to be adjusted for anyone trying the script. For me, the script rarely manages more than 8 repetitions.

import os
from pathlib import Path

TEMPFILE_DIR = Path(os.getenv("TEMP"))
DROPBOX_ROOT = Path("E:/Dropbox")      # Adjust for you system, usually C:/Users/Username/Dropbox"

def main():
    tempdirfile = TEMPFILE_DIR / "tmp.txt"
    dropboxfile = Path("E:/Dropbox/tmp/test.txt")

    i = 0
    while True:
        i += 1
        print(i)
        for file in (tempdirfile, dropboxfile):
            # file.unlink() # would solve in most cases, but still fails every few hundred attempts
            #               # also defeats the purpose of using the pattern in the first place
            tempdirfile.parent.mkdir(parents=True, exist_ok=True)
            tempfile = file.with_suffix(".txt.tmp")
            tempfile.write_text(TEXT)
            print("In {}: {} -> {}".format(file.parent, file.name, tempfile.name))
            tempfile.replace(file)

TEXT = """
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris justo
purus, pharetra quis ornare in, tincidunt et odio. Sed pulvinar lorem
vel justo vestibulum aliquet. Nunc et nisi dui. Cras tempus lacus ac
quam imperdiet mattis. Maecenas tristique dui vel orci condimentum
volutpat. Quisque ut mauris metus. Fusce libero metus, luctus eget
sagittis a, venenatis et ex. Aenean purus tellus, feugiat et libero
vel, congue elementum tellus. Cras at interdum nulla. Ut ut nunc id
nibh lacinia dapibus quis id turpis. Pellentesque habitant morbi
tristique senectus et netus et malesuada fames ac turpis egestas.
Vestibulum eu iaculis sapien. Fusce lobortis massa at molestie
posuere.
"""

if __name__ == "__main__":
    main()