You might see that the Dropbox Community team have been busy working on some major updates to the Community itself! So, here is some info on what’s changed, what’s staying the same and what you can expect from the Dropbox Community overall.
Forum Discussion
Architecture pattern used by Dropbox
As we have many software architecture patterns available like peer-to-peer, client server, Observer and a lot more, which one does Dropbox uses ? and why it has selected that specific architecture? P...
Hey Akhil_M_S, I'd happily take it from where you left off with Daphne; I hope my message finds you well today!
To answer your initial question Akhil, different features on different platforms tend to have slightly different models, but the most common pattern is the observer pattern. This is implemented as long-polling from the client to the server, which holds the canonical representation of the data. For example, we do this for a lot of sync features, but also for comments/file activity. Where we don't use long-polling, we do client/server polling.
Hope that helps with your class presentation and good luck with your project!
Hi Walter,
Am good. Hope all is well there.
Thanks for the quick reply. I appreciate that.
The reference diagram was helpful. That answer my question partially. My question was more towards the database. Based on the reference diagram,
- Block stroage servers - Is this something like Amazon S3?
- Metadata Database - I read in one of the blog ((Re)Introducing Edgestore), Dropbox uses MySql?
Is the above a right understanding?
I guess MySql was opted to maintain ACID. How is sharding done as scaling a RDBMS is a costly effort.
Thanks in advance.
Prasanth
Walter
Dropbox Staff
Dropbox StaffThanks for getting back to me Prasanth; how are you today?
I'll try to address your questions here but please feel free to also check our security whitepaper as most of your concerns are addressed within that document.
To cut to the chase now, note that (1) the actual contents of users’ files are stored in encrypted blocks with the Block Storage Servers while prior to transmission, the Dropbox client splits files into file blocks in preparation for the storage. The Block Storage Servers act as a Content-Addressable Storage (CAS) system, with each individual encrypted file block retrieved based on its hash value.
And (2) File metadata is stored in a MySQL-backed database service, and is sharded and replicated as needed to meet performance and high availability requirements.
And to further elaborate, file blocks are stored in one of two systems: Amazon Web Services (AWS) or Magic Pocket, our in-house storage system. Magic Pocket consists of both proprietary software and hardware and has been designed from the ground up to be reliable and secure. In both Magic Pocket and AWS, file blocks are encrypted at rest, and both systems meet high standards for reliability.
I hope this information helps and we're always around if you have any additional queries; just give us a nudge here prasanthkp!
Thanks Walter. I am doing good. How about you?
Your explanation helps. I appreciate that. I am reading the docs and will revert if I have more questions.
Cheers.
Prasanth
- Walter
Hi again Prasanth! I'm fine as well; thanks for asking.
I'm glad to hear the information I provided helps and should you come up with any additional Dropbox questions, please feel free to let us know.
Until we speak again, have a wonderful weekend prasanthkp!
Hi Walter,
Hope you are doing great.
I went through the security whitepaper. Thanks for sending this. It had lots of information. I would like to understand more about how you shard the MySQL db. As you mentioned below in the previous post.
"And (2) File metadata is stored in a MySQL-backed database service, and is sharded and replicated as needed to meet performance and high availability requirements."
I assume you use hash partitioning.
- How is sharding done?
- How edgestore routes to the correct "shard"?
- How is it replicated across different DC?
- Also is it still benefical to go with MySQL than a NoSQL db?
- Is there a detailed paper or blog around the sharding alone?
Thanks in advance.
Prasanth
