Prevent Drupal from Mistakenly Deleting Files with File Lock

Some time ago I was part of a team working on migrating an existing Drupal 6 site to Drupal 7. The content migration itself went smooth, but due to new features in the new version of the site, some manual work had to be done on the existing content. That's when we where faces with a weird behavior in Drupal - images used in the content started to disappear when we saved the nodes. They didn't just disappear from the nodes, they where physically deleted from the file system as well!

Automatic Deletion of Files

A bit of panic spread in the team and at first we couldn't figure out if this was because of something we did wrong with the migration or something else. After some digging and testing we found out that the problem we stumbled on is down to the way Drupal tries to clean up what it thinks are not used files. Basically what this means is that as soon as Drupal thinks a file is no longer used, it will automatically delete it without any warning. The file is then gone forever from the file system.

The reason this happen is due to the file_usage table. When you add a file using the file field, then the usage for that file gets an entry in the table, usage goes up with +1. However, when you delete a file, then that entry is deleted (-1). Drupal checks this constantly and as soon usage for a file is 0, then it will automatically delete the file from the system.

Note that I write adding files using the file field. If you use the Media module plugin for the CKEditor to embed files, those are not counted. This is because it is tricky to keep track of when files in WYSIWYG is removed as you have access to the source HTML. There is an issue discussion how this can be solved though.

There is no way for the user to prevent this happening and it can have really bad consequences for the site. Consider this common use case that many sites will have:

  1. User A creates a new node and adds a few images to the Media field.
  2. User B creates another node and uses the CKEditor plugin to find and embed one of user A's images. Now the same image is used and viewed on two different nodes.
  3. User A now edits his node and decides that the image no longer is needed and removes it.
  4. Drupal now notices that the image has a file usage of 0 and automatically deletes it.

Result: The image is deleted from the file system and the next time the cache is cleared it will not be visible on user B's node either.

Worse is that user A doesn't even have to save the node for this to happen. The file will be deleted as soon as the Remove media button is clicked.

There are many other use cases for when this is happening and I'm sure you can see how frustrating this will be for users since they will have no control over it without making sure the users are very careful about how they handle files for their content.

Luckily, until this is fixed in Drupal Core, there is now another solution to prevent this from happening.

File Lock

File Lock - prevent files from being deletedThanks to Sascha Riedinger (Shuairan on drupal.org) we can now use the File Lock module to prevent this behavior. What this module does is simply to make itself a user of files. That way file usage will never ever be 0 again and Drupal will never have the chance to happily delete them with headache and frustration for the users as result.

When installing this module the first thing you should do is to go to the admin/content/file page and use the batch operation to lock all the existing files on the site.

File Lock - prevent new files from being deletedThen go to admin/config/media/lock and configure how you want new files to be locked. You have several options available that gives you plenty of flexibility. Personally I have opted for the settings you see in the screenshot. That way I know that every file on my sites will be safe from deletion.

File Lock was just promoted to a full project on drupal.org and is currently only available in a dev version. However, it is working very fell and Sascha has done a great job creating a high quality module. It shouldn't be to long until we see a stable release of it.

Summary

The File Lock module will from now on be a standard module in the arsenal of modules I use on every site I build. Yes, it is not an optimal solution for this problem (needs to be changed in Core), but it is the best solution out there to prevent Drupal from automatically deleting files when it should not do it.

Personally I rather have a bunch of unused orphaned files laying around, than files mistakenly deleted when they should not.

Individuals:

Comments

walter's picture

Just in case if someone wonders if the media plugin for Tinymce behaves different regarding the file_usage count: No, it does not. You have exactly the same problem, when you use tinymce :/

I guess an alternative solution to the file locking is to use plain image fields for uploading, and the insert module (http://drupal.org/project/insert) to insert the images to wysiwyg fields.

And thanks pointing out this problem!

tsvenson's picture

Yes, you have the same issue with every WYSIWYG editor as far as I know.

Using Insert isn't an alternative to solve this. The problem is the same there. If you go back to a node that you have added a file with and removes it, it will be removed from the file system as well. Its just how core works currently.

What I do with the Media module is that I have a media field that I add the files I want to embed. That field is set to hidden in the display tab for the content type. Then you simply use the CKEditor plugin and the Views Library tab to find the files you just added. Yes, its a little extra work right now, but it works. Another advantage is that I can then take advantage of the fact that images are tied to the content they are used.

An Insert like feature will come to the Media module as well soon.

juan_g's picture

Related issues:

Make the file deletion in file_field_delete_file() optional
http://drupal.org/node/1399846

Remove Media at content edit page cause the file to be completely deleted from server/database?
http://drupal.org/node/1239056

WYSIWYG does not record file usage
http://drupal.org/node/1268116

File Lock (recently approved project application)
http://drupal.org/node/1381726

dalin's picture

You might want to put this at the top of the article:

TL/DR Bad bug in Drupal Core means that files get deleted at seemingly random times. Install this module to avoid the problem:
http://drupal.org/project/file_lock

tsvenson's picture

That would actually be telling a lie. Its not a bug, it is designed this way. Also, its not randomly since it happens every time a file is removed and file usage goes to 0.

Walt Daniels's picture

I assume this same problem occurs with IMCE.

tsvenson's picture

Actually no. IMCE is adding itself to the file_usage table for files you upload using it.

That is of course good, but it means you will have to upload all new files through IMCE and that will mean extrea steps in the workflow.

reverse phone lookup's picture

This is a permissions or security issue. The webserver runs as a special user. Any files it creates will be owned by that user. It does not have permission to modify any Drupal files, so a Web security breach can't be used to modify code. When Drupal creates files it needs to have write access to the files directory. Often that's done by giving the directory to the webserver, but that may lock your user account out. Thanks.

Add new comment