How to make Dropbox ignore node_modules folder with symbolic links (aliases)

[sebastian_]

PROBLEM

Dropbox does not currently provide an ignore file/folder feature that enables a locally-unique instance to exist on each machine. One important use of this is the “node_moduels" folder used for node javascript development. The node_modules folder sees thousands of file changes during the development cycle. If this folder is allowed to sync, the important files you need (outside of it), are so far down in the sync queue that you never have the important files when you need them on your other systems.

SOLUTION

A better way to sync projects with Dropbox while ignoring the node_modules folder is to use a “properly” formed symlink (alias). At this time npm and yarn support a property formed symlink. HOWEVER, IT IS CRITICAL THAT THE LINK (ALIAS) AND TARGET HAVE THE SAME NAME!

From shell prompt (terminal):

1. cd YOUR_PROJECT_NAME
2. mkdir -p ~/Downloads/node_modules_for_YOUR_PROJECT_NAME
3. mv node_modules ~/Downloads/node_modules_for_YOUR_PROJECT_NAME
4. ln -s ~/Downloads/node_modules_for_YOUR_PROJECT_NAME/node_modules

Now create the same target node_moduels folder on each of your other systems **AFTER** Dropbox syncs the ENTIRE project direcotry. The old node_modules folder should be removed and the symlink (alias) added before running these steps.

1. cd YOUR_PROJECT_NAME
2. mkdir -p ~/Downloads/node_modules_for_YOUR_PROJECT_NAME/node_modules
3. yarn install

SUCCESS

Dropbox WILL sync the symlink/alias file itself, but NOT the contents it points to. This seems to work well for file syncing across several of your own development systems, and then use git to push to the repo for others on the team.

CAVEAT

The tiny symlink (alias) file itself IS sync’d on all systems, but will point to a non-existent / empty folder on the other systems. This is most applicable for a single user, but can also be useful for teams if an individual variant of the ignored data is appropriate, such as for node_moduels.

Hope this helps.

🙂

..sebastian

[Terry U.1]

Ao I removed all node_modules folders on both my iMac and actually from within my Dropbox account.

However, since then my Dropbox won't complete, it has been on day after day and disabled sleep. It is stills tuck on 500K files. I had a very similar incident on my Macbook Pro when I removed a load of files via Selective Sycn and this too over a month to resolve, having to leave the machine on full time during this period without sleep. So, how can I resolve this.

At the moment, Dropbox is unusable. I am close to finding another provider who does sync files effectively, something that Dropbox was great at it, in it's early days. Now it seems to struggle with the basics of Syncing. Instead adding useless features like connecting Google Calendar and Outlook.... Why not focus on the core functionality of the syncing and ignoring of files. having been a paid subscriber since 2012 it is with disappointment and regret I am faced with having to look to another provider.

[qrayg]

I wrote an Apple Script for this specific problem (sorry Windows users):
https://gist.github.com/craigerskine/fe1278ca405d58cd61e349797f96d9d0

[fernandorojo]

Any shot you could upload a video showing how you did this? Slightly hard to follow with your usage of the Downloads folder. Thanks!

[devinrhode2]

Symlinks are probably the best general solution. For developers using Git, you have another option. If you are fanatic about creating great git history/commit log for your changes, then you should try using `git worktree add ../projectA-dupe` which will create a duplicate folder that's a sibling to your current git repo. It's actually another "working directory" in proper git terms. It's the same git repo, all the same branches, same stashes, all the git info is the same. Actually the .git dir for "projectA-dupe" basically just points to your original git repo's .git folder. The advantage here, the workflow, is to have your main working directory outside of dropbox entirely, and then add another "working directory" or "worktree" inside your dropbox folder, which will sync. Or, vice versa. You could have the main git repo be inside dropbox, but, setup an npm pre-install hook to fail installation if it sees "Dropbox" is in your current folder path. All developers are instructed to cd to the main git repo inside Dropbox, and then run `git worktree add ../../path/to/your/projectA`. Furthermore, you could have everyone keep their worktree inside dropbox, which means they could all collaborate without ever doing a git push/pull/fetch, by simply allowing dropbox to sync all the git objects. One developer could help another simply by cd'ing to their worktree inside of dropbox. They see the exact same thing, no need to stash changes, commit anything, etc. The symlink might as well be committed into the git repo, and point to a globally reliable path, like ~/universal_node_modules/projectA. Better than all that, probably use yarn v2, and inside .yarnrc.yml set cacheDirectory to be something like ~/Dropbox/.yarn-cache/projectA, so all worktree's can share the same cache. 

[sp1thas]

I've recently implemented dropboxignore which is a simple shell script which facilitates you to generate .dropboxignore files based on your file patterns or even based on existing .gitignore files and ignore matched files from dropbox. Hope to find it useful for your case. Any feedback is more than welcome. dropboxignore is currently available only for Mac OS and Linux.

[Riderpaul]

This is just dumb. No system should ever back up node_modules. This makes me want to rethink my Dropbox subscription.

[pdepedo]

It's 2021 and Dropbox still has no ignore functionality. I'm wondering how difficult would that be, really. Just before you sync a folder, check for existence of a .dropboxignore file, and then make sure you don't sync the files listed there. Like, seriously. How long can this take to implement? A week? A month? What the f are you waiting for? 

[rvvincelli]

Thank you so much! After so much time wasted fiddling with xattr andthe Dropbox selective sync!!

[Josh W.14]

I've fought with dropbox syncing node_modules, .git/, etc for years hoping the team would release a fix. I set up sym links, fs watchers to auto-set the ignore attribute for new folders, etc. To get it all working I have to turn off sync on all my computers, delete node_modules from dropbox.com, re-add the folder on each computer, re-add the ignore attribute, then re-enable syncing. Same for .git repos. If I ever need to blow away node_modules, I run `rm -rf node_modules/*`, otherwise I've got to redo the entire process. I've always loved Dropbox and always recommend it to family & colleagues, but last week I ran out of patience because all the Dropbox node_modules thrashing was destroying my IDE. I killed Dropbox & installed mega.io. It lets you setup a global ignore list where I added node_modules, .git, .idea, etc. It just works. I'll still recommend Dropbox for family, but for tech friends Dropbox is a nightmare. 

[Spl221]

I agree that if dropbox team does not support the users with must-have features for ages, I would rather cancel my subscription and use a better service. Could anyone recommend something that is aware about ignore patterns and has an UI for Linux/Windows machines?