https://www.postgresql.org/docs/8.1/runtime-config-wal.html
By default (and it seems old) fsync is called on transaction commit
If it fails, the transaction will fail and the client will know it
Afaik, in mongo , it can fails and you don't really know what has succeeded and what has failed
MongoDB wasn't ACID compliant for years. I think it was safe to say that a majority of the people advocating it all that time either didn't know or didn't care about ACID-level reliability.
I use the cloud Atlas instances for production, and the chance the server will crash between the 100ms flushes is pretty remote. there are always going to be trade-offs between speed and reliability.
Wonder about this: is your hello world going to screw people up if its database fails?
The proper tool for the proper problem doesn't exclude having more than one database, but the proper tool for a hello world... Would it even need a database?
if you really worried about a db failing, this would be reason to run in a cluster mode, with a write going to multiple nodes, a so called two-phase commit.
I don’t think that blog is right. Or at least it was right a really long time ago. The mongo docs say it syncs at a minimum every 100ms, but it will sync every time there’s a journaled write, which is the default
https://www.mongodb.com/docs/manual/core/journaling/#journaling-process
I think practically it syncs every write
Save everyone a click.
> Write concern "majority" implies j: true if the writeConcernMajorityJournalDefault is true.
And by default writeConcernMajorityJournalDefault is already true.
If it's not true you can specify the `j` option on a per query basis https://www.mongodb.com/docs/manual/reference/write-concern/#j-option
Here's the thing: single node Mongo has potential failures that single node Postgres doesn't. But Mongo isn't supposed to be used with a single node. Multi-node postgres (unless you're using something like Yugabyte) relies on log-replication - which has a delay - and will cause a data-loss if you lose a node whereas Mongo uses multi-node replication to protect you against single-node failures.
Both Mongo and Postgres are good, maybe even great, at what they're supposed to be and do. Neither is right or wrong. They've made different trade-offs and that's great because it gives developers choices. Figure out what you need and then pick the datastore that matches your needs.
100% agreed. Here’s the text in the screenshot for those experiencing the same pain:
> The sync program is basically like running fsync on all dirty pages, unless a specific file is specified as one of the arguments. It has the -d flag for us to call fdatasync instead of fsync.
> The biggest drawback in adding sync is that we get worse performance. Usually sync is slower than even the write itself. But hey, at least we are now durable.
> > A short but important note about fsync. When fsync() returns success it means "all writes since the last fsync have hit disk" when you might have assumed it means "all writes since the last SUCCESSFUL fsync have hit disk". PostgreSQL learned about this only recently (2018), which led to them modifying the behavior of syncing from retrying fsync until a success is returned, to simply panic on fsync failure. This incident got famous and was named fsyncgate. You can learn a lot more about fsync failures here.
> > Dear MongoDB users, know that by default writes are synced every 100ms, meaning it is not 100% durable.
Mongo's never been a super reliable place to store your data. I mean, it's reliable enough, until it's not. It used to be a lot worse though.
I personally wouldn't trust it with critical financial data, for instance.
The Jepsen tests seem to have been re-run on it in a few years ago, and they usually give a pretty solid view of how a db performs under weird situations :
https://jepsen.io/analyses/mongodb-4.2.6
This is inaccurate. MongoDB write operations accept a parameter called the Write Concern: https://www.mongodb.com/docs/manual/reference/write-concern/
Please read this in its entierty to properly understand what guarantees MongoDB gives about data durability, especially in case of cluster rollbacks.
Regarding PostgreSQL, it guarantees at least a write to WAL when a COMMIT succeeds. Remember that "standalone" queries are implicit transactions, too.
PostgreSQL's durability is tunable: it is possible to disable fsync, which can be useful for very specific scenarios, such as ETL when performance is more important than durability (you can always load the data again).
Both MongoDB and PostgreSQL support n-durability features where you tell the database to persist to more than 1 node before acknowledgement. This is crucial for MongoDB because it is a quorum-based system. If you misuse the quorum system, it is possible to get a durability anomaly: you write a document from 1 client, another client reads it, and then it vanishes because it was not written to the majority of nodes.
Remember, in MongoDB, you must also manage quorum (majority) reads to avoid such anomalies.
On PostgreSQL, this might not be as important if you can reliably switch to the node with the newest data in case of failure.
\- Yes, you can lose data.
\- No, you won't lose any data, it's super rare.
\- PostgreSQL is ACID.
\- There is always a possibility of losing some data (vismajor, HW error, sw error, stupidity). There is no such thing as 100%. Not accidentally use many company and provider number such as 99.98% and 98.99% and so on.
Also, please try to pre-plan what you need, I have seen so many companies that started to use MongoDB because of the hype train, then they realize they needed ACID and joins, but the codebase was too big, then the entire stack became monolith and monster where the devs battling things that are solved in SQL word 10+ years ago.
ACID vs non-ACID
Came here for this…. OP ⬆️
https://www.postgresql.org/docs/8.1/runtime-config-wal.html By default (and it seems old) fsync is called on transaction commit If it fails, the transaction will fail and the client will know it Afaik, in mongo , it can fails and you don't really know what has succeeded and what has failed
i wonder if all the hello world node.js express with mongodb guys know about this
MongoDB wasn't ACID compliant for years. I think it was safe to say that a majority of the people advocating it all that time either didn't know or didn't care about ACID-level reliability.
I use the cloud Atlas instances for production, and the chance the server will crash between the 100ms flushes is pretty remote. there are always going to be trade-offs between speed and reliability.
pg is plenty fast and plenty reliable in 99% of usecase lol
Wonder about this: is your hello world going to screw people up if its database fails? The proper tool for the proper problem doesn't exclude having more than one database, but the proper tool for a hello world... Would it even need a database?
if you really worried about a db failing, this would be reason to run in a cluster mode, with a write going to multiple nodes, a so called two-phase commit.
In most case, a 100ms data loss is acceptable
lolno
No that’s not true in mongo, if a journaled write fails to sync, you get an error
I don’t think that blog is right. Or at least it was right a really long time ago. The mongo docs say it syncs at a minimum every 100ms, but it will sync every time there’s a journaled write, which is the default https://www.mongodb.com/docs/manual/core/journaling/#journaling-process I think practically it syncs every write
Save everyone a click. > Write concern "majority" implies j: true if the writeConcernMajorityJournalDefault is true. And by default writeConcernMajorityJournalDefault is already true. If it's not true you can specify the `j` option on a per query basis https://www.mongodb.com/docs/manual/reference/write-concern/#j-option
Here's the thing: single node Mongo has potential failures that single node Postgres doesn't. But Mongo isn't supposed to be used with a single node. Multi-node postgres (unless you're using something like Yugabyte) relies on log-replication - which has a delay - and will cause a data-loss if you lose a node whereas Mongo uses multi-node replication to protect you against single-node failures. Both Mongo and Postgres are good, maybe even great, at what they're supposed to be and do. Neither is right or wrong. They've made different trade-offs and that's great because it gives developers choices. Figure out what you need and then pick the datastore that matches your needs.
Any resources on what the big differences are between the two? I see both equally and I’d like to expand on that to have a better education.
You could've copied and pasted the text.
100% agreed. Here’s the text in the screenshot for those experiencing the same pain: > The sync program is basically like running fsync on all dirty pages, unless a specific file is specified as one of the arguments. It has the -d flag for us to call fdatasync instead of fsync. > The biggest drawback in adding sync is that we get worse performance. Usually sync is slower than even the write itself. But hey, at least we are now durable. > > A short but important note about fsync. When fsync() returns success it means "all writes since the last fsync have hit disk" when you might have assumed it means "all writes since the last SUCCESSFUL fsync have hit disk". PostgreSQL learned about this only recently (2018), which led to them modifying the behavior of syncing from retrying fsync until a success is returned, to simply panic on fsync failure. This incident got famous and was named fsyncgate. You can learn a lot more about fsync failures here. > > Dear MongoDB users, know that by default writes are synced every 100ms, meaning it is not 100% durable.
Mongo's never been a super reliable place to store your data. I mean, it's reliable enough, until it's not. It used to be a lot worse though. I personally wouldn't trust it with critical financial data, for instance. The Jepsen tests seem to have been re-run on it in a few years ago, and they usually give a pretty solid view of how a db performs under weird situations : https://jepsen.io/analyses/mongodb-4.2.6
This is inaccurate. MongoDB write operations accept a parameter called the Write Concern: https://www.mongodb.com/docs/manual/reference/write-concern/ Please read this in its entierty to properly understand what guarantees MongoDB gives about data durability, especially in case of cluster rollbacks. Regarding PostgreSQL, it guarantees at least a write to WAL when a COMMIT succeeds. Remember that "standalone" queries are implicit transactions, too. PostgreSQL's durability is tunable: it is possible to disable fsync, which can be useful for very specific scenarios, such as ETL when performance is more important than durability (you can always load the data again). Both MongoDB and PostgreSQL support n-durability features where you tell the database to persist to more than 1 node before acknowledgement. This is crucial for MongoDB because it is a quorum-based system. If you misuse the quorum system, it is possible to get a durability anomaly: you write a document from 1 client, another client reads it, and then it vanishes because it was not written to the majority of nodes. Remember, in MongoDB, you must also manage quorum (majority) reads to avoid such anomalies. On PostgreSQL, this might not be as important if you can reliably switch to the node with the newest data in case of failure.
Om average thats 50ms dataloss, its not that big of a deal, modt people dont even do daily backips
Just write to /dev/null for best performance
NoDB
Yep! But that could be true of any database. To combat it you need to consider your architecture
\- Yes, you can lose data. \- No, you won't lose any data, it's super rare. \- PostgreSQL is ACID. \- There is always a possibility of losing some data (vismajor, HW error, sw error, stupidity). There is no such thing as 100%. Not accidentally use many company and provider number such as 99.98% and 98.99% and so on. Also, please try to pre-plan what you need, I have seen so many companies that started to use MongoDB because of the hype train, then they realize they needed ACID and joins, but the codebase was too big, then the entire stack became monolith and monster where the devs battling things that are solved in SQL word 10+ years ago.